# Backtest Overfitting: Walk-Forward Tests and Point-in-Time Data Source: https://finzdata.com/learn/overfitting-walk-forward-point-in-time Why backtests overfit, and how a hidden hold-out, walk-forward testing, point-in-time data and a deflated Sharpe ratio stop overfitting before it costs you. # Backtest Overfitting: How Walk-Forward Testing and Point-in-Time Data Fix It Most backtests look better than the strategies they describe. The cause is rarely fraud. It is overfitting: a strategy shaped, step by step, to the quirks of one historical sample until it describes the past perfectly and the future poorly. This guide explains where overfitting comes from and the four tools that stop it: a deflated Sharpe ratio, walk-forward testing on unseen data, point-in-time data and a hidden hold-out. ## Where overfitting comes from Overfitting has three common sources. **Too many trials.** Every parameter change, filter or discarded idea is a trial. Test enough variants and one will look excellent by chance. If you only report the winner, the result says more about how hard you searched than about the market. **Data that knows the future.** Economic data is revised. Companies are delisted. Index members change. A backtest on today's version of history sees revised numbers that were not available at the time and leaves out companies that failed. That is look-ahead and survivorship bias, and it inflates almost every result. **Fragile rules.** A strategy that only works with a lookback of exactly one value has learned noise, not behaviour. AI makes all three worse. A model can generate and test hundreds of variants in minutes, and it often works from data it remembers rather than data as it was known. In the FinzData AI Macro Tournament Round 1, 12 AI models agreed with each other 84% of the time yet were right on only 29 of 83 calls. Agreement is not evidence. ## Fix 1: Count every trial with a deflated Sharpe ratio A raw Sharpe ratio treats each backtest as the only idea you ever had. The deflated Sharpe ratio raises the bar as the number of trials grows and accounts for skewed, fat-tailed returns. FinzData Alpha Factory requires a deflated Sharpe ratio of at least 0.95 at the true trial count, and the trial count comes from a session ledger that records every attempt, not from memory. ## Fix 2: Walk forward on unseen data Walk-forward testing builds a strategy on one period and judges it on the next, which it has never seen. The idea is simple: if the edge is real, it should carry into data the strategy was not fitted to. Alpha Factory requires every candidate to beat its benchmark on unseen validation data before it can go further. Two related checks catch fragile rules. The strategy must stay profitable at 2x costs, and it must survive a change of plus or minus 20% on every parameter. It also needs at least 100 trades or at least 5 years of evidence, so a handful of lucky trades cannot carry it. ## Fix 3: Use point-in-time data Walk-forward testing only works if each period shows what was actually known at the time. Point-in-time data keeps every vintage of a macro series, includes delisted companies and records fundamentals as filed. Alpha Factory runs on FinzData's point-in-time history: 16,000+ US stocks since 2000 including delisted names, 5,100+ ETFs, 123 indexes, 139 futures markets with every contract since 2007, end-of-day options chains since 2010 for 5,800+ underlyings with point-in-time dealer gamma, 22,700+ macro series with every vintage kept, COT positioning since 1986 and SEC fundamentals as filed. ## Fix 4: A hidden hold-out that opens once Even careful researchers peek. Each look at the test data is a small trial that leaks information back into the strategy. Alpha Factory holds back the latest 24 months. After a strategy clears every other gate, that hold-out opens once and decides. There is no second look, and nothing is quietly re-run. Failures are published too. The last gate keeps the published set useful: a new strategy must have a correlation below 0.8 to anything already published. See the full process on the [Alpha Factory research page](https://finzdata.com/research/alpha-factory-ai-trading-research-engine). ## Run an overfitting check with your AI Alpha Factory is a research engine, not a trading bot, and it never trades your money. It works with Claude, ChatGPT, Grok, Gemini, Perplexity, Qwen, Kimi, GLM, Llama and local models through the FinzData MCP server and REST API. [Connect your AI](https://finzdata.com/mcp) and try: Use FinzData Alpha Factory to find a /ES strategy that survives 2022, and tell me if it is real. Access is free. Start without a key on the last 6 months of data with 20 trials a day, or open the last 12 months with a free email account. ## FAQ ### What is overfitting in backtesting? Overfitting is when a strategy is tuned so closely to one historical sample that it captures noise rather than a repeatable effect. It shows up as a strong backtest followed by weak live results. ### What is walk-forward testing? Walk-forward testing builds a strategy on one period and judges it on a later period it has never seen, then repeats. It checks whether an edge carries into unseen data. ### Why does point-in-time data matter? Without it, a backtest uses revised figures and leaves out delisted companies, so it trades on information nobody had at the time. Point-in-time data shows the market as it was known on each day. Learn more on the [Alpha Factory research page](https://finzdata.com/research/alpha-factory-ai-trading-research-engine). This material is for information only and is not investment advice. ## Related - [Can AI Find a Profitable Strategy? AI Trading, Tested Honestly](https://finzdata.com/learn/ai-trading-strategy-test) - [Best AI Trading Bot? Test the Strategy Behind It First](https://finzdata.com/learn/best-ai-trading-bot-checklist) - [How to Backtest with Claude or ChatGPT on Point-in-Time Data](https://finzdata.com/learn/backtest-with-claude-chatgpt) - [FinzData Alpha Factory: AI Trading Research & Backtesting Engine](https://finzdata.com/research/alpha-factory-ai-trading-research-engine) - [Deflated Sharpe Ratio Explained](https://finzdata.com/learn/deflated-sharpe-ratio) - [Look-ahead bias](https://finzdata.com/learn/look-ahead-bias) - [Survivorship bias](https://finzdata.com/learn/survivorship-bias) - [Why backtests lie](https://finzdata.com/research/why-backtests-lie) - [AI Macro Tournament 2026](https://finzdata.com/research/ai-macro-tournament-2026) - [Connect your AI](https://finzdata.com/mcp)