Why your backtest is probably lying
Run enough coin flips and one of them will look like a magic coin. Run enough backtests and one of them will look like a genuine trading edge — even if every single one of them was pure noise. This is the multiple-testing problem, and it is the single most common way backtests deceive the people who ran them.
The Deflated Sharpe Ratio (DSR), introduced by Bailey & López de Prado in "The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality" (2014), is a statistic built specifically to answer one question honestly: given how many variants you tried before landing on this one, how much of your Sharpe ratio is real, and how much is the expected result of searching?
The core idea
If you test one strategy, a Sharpe ratio of 1.5 is meaningfully different from a Sharpe ratio of 0. But if you test 200 strategies and keep the best one, some of those 200 will show a high Sharpe ratio purely by chance — that is simply what happens when you take the maximum of many random draws. The more variants you try, the higher the bar climbs for a result to still look surprising. DSR raises that bar explicitly and tells you whether your result clears it.
The formula, symbol by symbol
1. The Sharpe ratio estimator's uncertainty
A Sharpe ratio computed from a finite sample isn't a fixed truth — it is an estimate with its own uncertainty, and that uncertainty depends on the shape of the return distribution, not just its mean and variance.
PSR(SR*) = Φ( (SR̂ − SR*) · √(T − 1) / √(1 − γ₃·SR̂ + ((γ₄ − 1)/4)·SR̂²) )
- PSR(SR*) — the Probabilistic Sharpe Ratio: the probability that the strategy's true Sharpe ratio exceeds a benchmark
SR*, given everything observed. - SR̂ — the observed Sharpe ratio, always per period (e.g. daily), never annualized. Annualizing before this formula is one of the most common ways people silently corrupt the result.
- SR* — the benchmark you're testing against. Naively, people use 0 ("is this better than nothing?"). DSR instead uses a benchmark that already accounts for how many things you tried — see below.
- T — the number of return observations in the backtest (e.g. trading days).
- γ₃ — the sample skewness of the return series. Negative skew (occasional large losses) makes the Sharpe ratio estimator less reliable and PSR more conservative.
- γ₄ — the sample kurtosis of the return series, in the non-excess convention where a normal distribution has kurtosis 3 (not 0). Fat tails inflate this number and further discount the result.
- Φ — the standard normal cumulative distribution function, converting a z-score into a probability.
2. The benchmark that accounts for how much you searched
SR*₀ = √V · [ (1 − γ)·Φ⁻¹(1 − 1/N) + γ·Φ⁻¹(1 − 1/(N·e)) ]
- N — the total number of independently tested strategies or variants, including this one. This is the number that must be tracked honestly and counted before seeing results — inflating it after the fact defeats the entire purpose.
- V — the variance of the (per-period) Sharpe ratios across all N tried strategies. If you don't know it, a standard conservative approximation is available (see the calculator) — but a measured V from your own actual trial history is always better.
- γ — the Euler–Mascheroni constant (≈ 0.5772), part of the asymptotic formula for the expected maximum of N draws from a normal distribution.
- e — Euler's number.
- Φ⁻¹ — the inverse of the standard normal CDF (the quantile function).
SR*₀ is the expected value of the best Sharpe ratio you'd see among N strategies if every single one of them had zero true skill. It rises as N grows — the more you search, the higher a result needs to be before it stops looking like the winner of a lottery.
3. Putting it together
DSR = PSR(SR*₀)
The Deflated Sharpe Ratio is simply the Probabilistic Sharpe Ratio evaluated against that inflated, search-aware benchmark instead of against zero. A DSR of 95% is often read as "still standing at the 95% confidence level after accounting for everything you tried."
Common ways people get this wrong
- Using an annualized Sharpe ratio directly in the formula. The formula's √(T−1) scaling assumes a per-period Sharpe. Plugging in an annualized number silently produces a nonsensical result.
- Not counting every variant. If you tried 40 parameter combinations and only remember testing "a few," your real N is 40, not "a few." DSR is only as honest as the trial count you feed it.
- Confusing excess and non-excess kurtosis. A normal distribution has kurtosis 3 in the non-excess convention this formula uses, and 0 in the excess convention many software packages default to. Mixing them up silently changes the result.
- Combining several individually-failed strategies and testing only the combination. A combination is its own new hypothesis with its own N contribution — it does not inherit a pass from strategies that individually failed, and testing only the combination while ignoring its own place in the trial count reintroduces exactly the bias DSR exists to catch.
What DSR does not tell you
DSR corrects for selection bias and non-normality within the sample you tested on. It does not verify that your backtest itself is free of look-ahead bias, survivorship bias, unrealistic transaction costs, or that the market regime you tested in will persist. A high DSR is necessary evidence, not sufficient proof.
A worked example from my own research
I ran a research project where I committed to a fixed methodology from start to finish:
- I pre-registered 109 hypotheses, each with a locked approval criterion, before seeing any results.
- I corrected every result using the Deflated Sharpe Ratio.
- I used git history to log everything and to make sure the methodology was actually followed.
The DSR analysis gave an acceptance rate of 19 out of 109. I wasn't satisfied with stopping there, so I went back and audited how I had actually worked. I found that many hypotheses had been built on top of hypotheses that were already approved. In other words, the tests were not independent — and independence is an assumption DSR relies on.
To handle this, I re-ran the analysis for the 19 DSR-approved hypothesis with a different multiple-testing correction, alpha-investing, which explicitly takes into account the order in which the tests were run. 18 of the 19 held up; one hypothesis (HYP-053) was rejected.
My takeaway: even a careful researcher is fully capable of tripping up themselves and their own project. The number of trials is only part of the truth.
Educational tool only — not investment advice, not a buy/sell signal. Past performance does not predict future results. No promise of returns is made anywhere on this site.