Survivorship Bias Free Stock Data Download

Understanding Survivorship Bias in Backtests

Survivorship bias arises when historical analysis excludes securities that are no longer trading, such as delisted or bankrupt companies. This leads to overly optimistic results because poor performers are omitted from the dataset.

For example, a backtest of a momentum strategy from 2000 to 2010 that only includes today's S&P 500 constituents would miss companies like Lehman Brothers or Enron, which were once large-cap but later delisted.

FinzData eliminates this bias by providing historical data for both active and delisted stocks, with delisted symbols marked by the '-DELISTED' suffix.

Accessing Delisted Stock Data

Delisted stock data is available via the FinzData Python client using the standard Ticker interface. Symbols for delisted names include the '-DELISTED' suffix, such as 'LEHMQ-DELISTED' for Lehman Brothers.

To retrieve historical bars for a delisted stock, use the history method with period='max' to access the full available history since 2000. This requires a look-back bundle or Institutional plan, as free access is limited to the last 12 months.

The returned DataFrame includes Open, High, Low, Close, and Volume columns indexed by date, enabling full historical analysis.

import finzdata as yf

# Retrieve full history for a delisted stock (requires look-back bundle or Institutional)
ticker = yf.Ticker("LEHMQ-DELISTED")
history = ticker.history(period="max")
print(history.head())
print(f"Total bars: {len(history)}")

Building a Point-in-Time Universe

A point-in-time universe includes only stocks that were active and had available data on a specific past date, preventing look-ahead bias. This is essential for realistic backtesting.

To construct such a universe, you must determine which stocks had valid price data on each rebalancing date, including delisted names that were still trading at that time.

FinzData's daily bar data, available via c.prices() with adjust='all', allows you to check for data presence on any date within the look-back period.

Example: Checking Stock Availability on a Specific Date

To verify if a stock had data on a given date, query its historical bars and check for the presence of that date in the index. This approach works for both active and delisted stocks.

For example, to check if Lehman Brothers was trading on September 1, 2008, retrieve its history and test for that date. If the date exists in the index, the stock had valid data and should be included in the universe for that date.

This method ensures that your universe reflects only the information available at the time, avoiding look-ahead bias.

import finzdata as yf
import pandas as pd

# Check if a delisted stock had data on a specific date
def has_data_on_date(symbol, date_str):
    ticker = yf.Ticker(symbol)
    hist = ticker.history(period="max")
    return date_str in hist.index.strftime('%Y-%m-%d')

# Example: Check Lehman Brothers on Sept 1, 2008
date_to_check = "2008-09-01"
symbol = "LEHMQ-DELISTED"
if has_data_on_date(symbol, date_to_check):
    print(f"{symbol} had data on {date_to_check}")
else:
    print(f"{symbol} did not have data on {date_to_check}")

Assembling a Monthly Rebalancing Universe

To build a monthly rebalancing universe, iterate over your backtest period and, at each month-end, collect all stocks with valid data on that date. This includes filtering by exchange (NYSE/Nasdaq) and applying any additional filters like minimum price or volume.

FinzData does not provide a direct list of all symbols by exchange, but you can maintain a static list of active and delisted symbols (approximately 16,000 stocks and 231 ETFs) and filter it based on data availability at each point in time.

For each candidate symbol, use the has_data_on_date function to determine if it should be included in the universe for that rebalancing date.

import finzdata as yf
import pandas as pd
# Needs a look-back bundle or the Institutional plan (free keys get HTTP 403)

symbols = ["AAPL", "MSFT", "LEHMQ-DELISTED", "ABK-DELISTED"]
universe_by_date = {}

for month_end in pd.date_range(start="2008-01-31", end="2008-12-31", freq="M"):
    date_str = month_end.strftime('%Y-%m-%d')
    available_stocks = []
    for symbol in symbols:
        try:
            hist = yf.Ticker(symbol).history(start="2000-01-01", end="2012-12-31")
            if date_str in hist.index.strftime('%Y-%m-%d'):
                available_stocks.append(symbol)
        except Exception:
            continue
    universe_by_date[date_str] = available_stocks
    print(f"{date_str}: {len(available_stocks)} stocks available")

Avoiding Common Pitfalls

A common mistake is assuming that a stock's delisted symbol will work in all API calls without adjustment. Always verify the exact symbol format, which includes the '-DELISTED' suffix as shown in FinzData's stock pages.

Another pitfall is using split-only or dividend-only adjusted prices when calculating returns. For survivorship bias-free analysis, use split-and-dividend adjusted data (adjust='all') to ensure corporate actions do not create artificial gaps or jumps in the price series.

Finally, remember that free API access is limited to the last 12 months of daily bars. To access historical data for delisted stocks or build multi-year universes, a look-back bundle or Institutional plan is required.

Validating Your Survivorship Bias-Free Universe

To validate that your universe avoids survivorship bias, confirm that it includes known delisted stocks on dates when they were still trading. For example, Lehman Brothers should appear in your universe for dates prior to its September 2008 delisting but not after.

You can also check that the count of stocks in your universe fluctuates over time, reflecting real-world changes due to listings, delistings, M&A activity, and bankruptcies. A static universe size over decades would indicate a bias.

Cross-checking with historical index membership data (e.g., past Russell 1000 or S&P 500 constituents) can provide additional validation, though FinzData does not currently offer this as a direct feed.

import finzdata as yf

# Validate Lehman Brothers inclusion before and after delisting
def check_symbol_in_universe(symbol, date_str):
    ticker = yf.Ticker(symbol)
    hist = ticker.history(period="max")
    return date_str in hist.index.strftime('%Y-%m-%d')

symbol = "LEHMQ-DELISTED"
print(f"{symbol} in universe on 2008-08-31: {check_symbol_in_universe(symbol, '2008-08-31')}")
print(f"{symbol} in universe on 2008-10-31: {check_symbol_in_universe(symbol, '2008-10-31')}")