This guide explains how overfitting happens, the warning signs, the tests that catch it, and the habits that keep strategies robust.
How overfitting happens
Every strategy has settings: entry times, indicator periods, stop levels, filters. Each setting you tune gives the strategy another way to fit the past. With enough settings, a strategy can be made to fit almost any history, including patterns that were pure chance.
Consider testing 100 versions of a strategy on the same data. Even if none of them has a real edge, some will look good by luck. If you choose the best one and trust its backtest, you have selected luck, not skill. The more versions you test, the better the best one will look, and the less its result means.
Researchers have shown how strong this effect is. With five years of daily data, trying more than about 45 independent strategy variations is almost guaranteed to produce one with an attractive in-sample Sharpe ratio even if none has any real edge.
Warning signs of an overfitted strategy
- Many conditions. Five filters and three exceptions to make the equity curve smooth.
- Precise, unusual settings. An RSI of 13 and a stop of 37 points, when 14 and 35 fail.
- Neighbours that fail. Small changes to a setting turn a profit into a loss.
- Results concentrated in a few trades or one period.
- A smooth equity curve with very few losing months, far better than similar strategies.
- No clear reason why the strategy should work.
Tests that catch overfitting
Out-of-sample testing
Split your history into two parts. Design and tune the strategy on the first, then test it once on the second, which it has never seen. If performance collapses, the strategy was fitted to the first part. The test only works if you do not go back and tune on the second part afterwards.
Walk-forward testing
Repeat the out-of-sample test over time. Design on one window, test on the next, then move forward and repeat. The combined test periods give a fairer estimate of live performance. See walk-forward testing.
Parameter stability
Run the strategy across a grid of nearby settings. A robust strategy shows a broad region of reasonable results. An overfitted one shows a single sharp peak.
| Stop loss | 20% | 25% | 30% | 35% |
|---|---|---|---|---|
| Robust strategy | Profit | Profit | Profit | Small profit |
| Overfitted strategy | Loss | Loss | Large profit | Loss |
This is an illustration. The robust strategy works across its neighbourhood; the overfitted one works only at one setting.
Different markets and periods
A real edge should appear, at least partly, in related markets or other periods. A Nifty strategy that also works on Sensex, or in earlier years, is more likely to be robust.
Monte Carlo simulation
Reshuffle the order of trades thousands of times to see the range of drawdowns the same trades could have produced. See Monte Carlo simulation.
Adjust for the number of trials
Statistical methods such as the deflated Sharpe ratio adjust a backtest's performance for how many versions were tried. Even without the maths, keep a record of every version you test and be most sceptical of the best one.
Habits that keep strategies robust
- Start with a reason. Write down why the strategy should work before testing it.
- Keep rules few. Each extra condition should earn its place across many periods.
- Use round, common settings unless there is a reason for something else.
- Fix the test plan in advance: the data, the period, the metrics and what counts as success.
- Hold back recent data for a final test.
- Count trades. Results based on fewer than 100 trades are fragile.
- Paper trade before going live, and compare the results with the backtest.
A hypothetical example
A trader tests a Nifty straddle with 6 entry times, 5 stop levels, 4 re-entry rules and 3 day filters: 360 combinations. The best shows a high return with a tiny drawdown. On the following year, which the trader did not use, it loses money.
A second trader tests only 3 entry times and 3 stop levels, chosen in advance, and finds that 8 of the 9 combinations are profitable, with the best and worst not far apart. On the following year, results are lower than the backtest but still positive. The second strategy is less impressive on paper and far more trustworthy.
Overfitting and automation
Automation makes testing fast, which makes overfitting easier. A platform that can run hundreds of backtests in minutes needs discipline from the user. QuantPower's backtesting engine shows every trade and lets you compare settings; use it to test robustness across neighbours and periods, not just to find the best curve. Then run the strategy in paper trading before deploying it from the strategy builder.
Start your free trial
Connect your broker, build a strategy and test it on paper before a single live order goes out.
Frequently asked questions
What is overfitting in trading?
Tuning a strategy so closely to past data that it captures random noise. It performs well in the backtest and poorly on new data.
How do I know if my strategy is overfitted?
Check whether nearby settings also work, whether it holds up on data it was not designed on, and whether its results depend on a few trades.
Is optimisation always overfitting?
No. Choosing reasonable settings is part of design. It becomes overfitting when you search many combinations and trust the best one without testing it on new data.
How much data do I need to avoid overfitting?
Enough for many trades across different market conditions, plus a separate period for out-of-sample testing. More rules and settings need more data.
Sources
- Bailey, D. H., Borwein, J. M., López de Prado, M. and Zhu, Q. J. (2014). Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society, 61(5), 458–471
- Bailey, D. H. and López de Prado, M. (2014). The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality. Journal of Portfolio Management, 40(5), 94–107
- Bailey, Borwein, López de Prado and Zhu: The Probability of Backtest Overfitting