This plot shows maximum Sharpe ratios (a widely used financial performance metric), based on pseudorandom trials where the true Sharpe ratio is zero, as a function of the number of trials. Results with higher probability are cast in a lighter color. The dashed line are the results predicted by the False Strategy theorem. The False Strategy theorem tells us that with enough trials, there is no Sharpe ratio threshold sufficiently high enough to reject the hypothesis that a strategy is false. Given the ease with which one can use a computer to explore many variations of a fund or strategy and only select the optimal variation, it follows that it is very easy to produce impressive-looking designs that are nothing more than false positives. This is the essence of backtest overfitting, which is an instance of the larger problem of selection bias under multiple testing, a problem that afflicts finance and numerous other fields of science and technology.

Overview

We are also concerned with the proliferation of pseudomathematical investment advice and financial columns in the past few years, which appear to be based on sophisticated mathematics and statistics, but which, upon more rigorous analysis, are at best questionable. We encourage the reader to search the Internet for terms such as "technical indicators," "technical analysis" "Elliott waves," "Fibonacci ratios," "Golden ratios," "parabolic SAR," "pivot point," "momentum," "stochastic oscillators," and others in the context of finance. Although such terms clearly evoke precise mathematical concepts, in fact, in almost all cases, their usage is scientifically unsound.

It is particularly disappointing that such pseudomathematical language and techniques are being promoted by leading financial news organizations and brokerage houses. Again, the reader is invited to log in to his or her own brokerage service and search for terminology such as those listed in the previous paragraph in the broker's analysis of various stocks, bonds, mutual funds and exchange-traded funds. The fact that these brokers have a vested interest in their customers' frequent trading makes such commentary particularly galling.

Historically scientists have led the way in exposing those who utilize pseudoscience to extract a commercial benefit. In the 18th and 19th century, physicists exposed the nonsense of astrologers, and beginning in the 20th century governments started requiring strict double-blind scientific tests, with rigorous statistical standards, of all pharmaceutical products. Yet mathematicians in the 21st century have remained disappointingly silent with the regards to those in the investment community who, knowingly or not, misuse mathematical techniques such as probability theory, statistics and stochastic calculus.

Our silence is consent, making us accomplices in these abuses.

The Mathematical Investor blog and other materials accessible from this website were established with these concerns in mind. Our research and writings focus on ways to better understand and mitigate these difficulties and ways to assist other professionals in the field, in addition to rigorous, unbiased testing and analysis. If you identify with our concerns, let us know and spread the word. Together we can make a difference.

Mailing list

General information

• David H. Bailey, Lawrence Berkeley National Laboratory (retired).

Please send any comments or questions for this site to:

Mathematical Investor blog

• Mathematical Investor blog

Online tools and videos

• Backtest overfitting demonstration
• Tenure-maker overfitting demonstration
• Selection bias video

Disclaimer and copyright