FinzData

Deflated Sharpe Ratio: Is Your Backtest Real or Overfit?

The deflated Sharpe ratio is a statistical test that adjusts a strategy's Sharpe ratio for how many variants you tried and for how non-normal its returns are. It tells you the probability that your backtest result reflects a real edge rather than luck after repeated searching.

The multiple testing problem

Backtest overfitting happens quietly. You test a moving-average crossover, tweak the lookback, add a filter, drop a sector, try a different exit — and after enough iterations one configuration looks excellent. Each of those attempts is a trial, and the more trials you run, the more likely it is that the best result is simply the luckiest draw from noise, not a repeatable edge.

A raw Sharpe ratio ignores all of this. It measures one backtest in isolation, as if it were the only idea you ever had. That is why a high Sharpe on its own is weak evidence: it says nothing about how hard you searched to find it.

From Sharpe to probabilistic Sharpe to deflated Sharpe

The probabilistic Sharpe ratio asks a first correction: given the length and variability of the track record, what is the probability that the true Sharpe is above zero? It accounts for estimation error and for skewed, fat-tailed returns, which make an observed Sharpe less reliable than the textbook formula assumes.

The deflated Sharpe ratio (Bailey and López de Prado, 2014) goes one step further. It raises the hurdle the observed Sharpe must clear based on the number of independent trials behind it. Test one strategy and the hurdle is modest; test hundreds of variants and the same Sharpe may no longer be statistically meaningful. This is the standard answer to the probability of backtest overfitting: not a gut feeling, but a number.

Why counting trials honestly matters

The hardest input is n_trials. It is not just the backtests you saved — it is every variation you evaluated and acted on, including the ones you discarded after a quick look. Undercounting trials flatters the result; overcounting is conservative but safe. A practical rule: count every distinct configuration whose performance influenced your next decision.

Because the trial count is so easy to fudge when you compute it yourself, it helps when the environment counts for you. If every run goes through one tool, the tool can keep the tally.

How FinzData reports it

FinzData's server-side run_backtest tool — available through the remote MCP endpoint at https://api.finzdata.com/mcp — returns a deflated Sharpe report with every run. It includes n_trials (the number of distinct strategy specs your API key has backtested), the dsr value, and a plain-language verdict:

  • "likely real" — dsr ≥ 0.95
  • "inconclusive" — dsr ≥ 0.5
  • "likely overfit" — dsr below 0.5
import finzdata as fd

prices = fd.Ticker("AAPL").history(interval="1d", start="2010-01-01")
result = fd.run_backtest(prices, strategy="sma_cross", fast=20, slow=100)
print(result["n_trials"], result["dsr"], result["verdict"])

Because the data behind the backtest is survivorship-free and point-in-time, the Sharpe being deflated is itself free of survivorship and look-ahead contamination — the DSR then isolates the remaining risk, which is your own search process.

FAQ

What is the deflated Sharpe ratio?

The deflated Sharpe ratio (Bailey and López de Prado, 2014) adjusts a strategy's Sharpe ratio for the number of strategy variants tried and for non-normal returns. It answers: given how many things I tested, what is the probability that this Sharpe is genuinely above zero rather than the best of many random attempts?

Why does counting trials matter in backtesting?

Every parameter sweep, feature tweak and discarded idea is a trial. The more trials you run, the more likely it is that one variant looks good purely by chance. The deflated Sharpe ratio raises the bar a strategy must clear as the trial count grows, so an honest count of trials is essential.

How does FinzData report the deflated Sharpe ratio?

FinzData's server-side run_backtest tool returns n_trials (the number of variants tried), the dsr value and a plain verdict: "likely real" when dsr is at least 0.95, "inconclusive" when it is at least 0.5, and "likely overfit" otherwise.

What is the probability of backtest overfitting?

The probability of backtest overfitting is the chance that a strategy selected for its strong in-sample performance has no real edge and will disappoint out of sample. The deflated Sharpe ratio framework quantifies this by penalising the observed Sharpe for multiple testing and for skewed, fat-tailed return distributions.