Five silent reasons the same trading strategy gives different results across backtesters
Port a simple systematic strategy from one backtester to another and the results almost never match on the first try. Often the trade count is off by 15%, the drawdown shifts by several percentage points, and the Sharpe ratio moves enough to change your mind about trading it.
Most of the time, neither backtester has a bug. They simply made five default choices differently. Because those choices compound over thousands of trades, a small rule difference in year one produces a different portfolio by year five. (This note focuses on daily portfolio strategies; intraday bars add session boundaries, extended hours and start- vs. end-of-bar timestamp conventions as well.)
Here are the five mechanics that cause most divergences, with concrete places they show up.
1. Signal-bar fills vs. next-session fills
Suppose your rule buys a stock when its 2-day RSI closes below 10.
- Same-bar close fill: A tool that defaults to filling on the signal bar executes at that exact close. That assumes you knew the final closing print, ranked every candidate across the universe, sized the orders from final closing equity and filled at that same close with zero gap.
- Next-session open fill: An engine that evaluates rules at the close and fills at the next session’s open (
next_open) absorbs the overnight gap.
For mean-reversion strategies in particular, the overnight gap between the signal close and the next morning’s open is often a large share of the trade’s return. Filling on the signal bar inflates backtest returns and produces orders you cannot actually place in an evening routine. (bt, for example, fills at the decision close by default, whereas Rearview defaults to fill next_open and marks fill same_close runs as approximate.)
2. Pre-adjusted prices vs. as-traded prices
This is the most common source of distortion when importing free per-symbol CSV files.
Suppose a stock traded at $20 in 2016, later split 4-for-1 twice (16-for-1 total, pulling the split-adjusted 2016 price to $1.25), and paid roughly a 1.3% annual dividend yield over the next decade (about a 0.88 multiplicative dividend factor). In a vendor’s back-adjusted CSV, that 2016 close shows up around $1.10.
Scale-invariant indicators such as rsi, roc or close > sma(close, 200) are unaffected by multiplicative split adjustment. Three other things break immediately:
- Fixed-dollar price filters leak future outcomes. A rule requiring
close > $5skips stocks that were actually trading at $20 in 2016. Worse, backward split adjustment introduces subtle look-ahead into price filters: future big winners that forward-split get pushed below $5 historically (excluded), while future losers that reverse-split get pushed above $5 (included). - Share counts and per-share commissions distort. Buying $10,000 of a stock at $1.10 buys
floor(10,000 / 1.10) = 9,090shares instead of500shares. At a$0.005/sharecommission, you get charged$45.45instead of$2.50: an 18.2x commission error on a single entry. - Dollar-volume filters break if volume is not inverse-adjusted. Dividing price by 16 without multiplying volume by 16 shrinks historical dollar volume by 16x.
To avoid all three, an engine needs to store as-traded prices alongside separate split and dividend tables, and compute three distinct views using only past actions:
- As-traded prices for fills, price filters (
close > $5), share counts and per-share commissions. - Split-adjusted prices for price averages and levels (
sma,highest,atr) so a 2-for-1 split does not look like a 50% crash. - Total-return prices (splits plus reinvested dividends) for return-based and percentage indicators so an ex-dividend drop does not trigger a false dip buy.
3. Survivorship bias and delisting haircuts
Two backtests on "S&P 500 stocks from 2010 to 2025" are often testing different markets:
- Today’s constituents vs. point-in-time membership. Running a rule on today’s 500 tickers guarantees every stock in your 2010 universe survived the next 15 years. Point-in-time membership only allows trading stocks that were actually in the index on each historical date.
- What happens when a stock delists. Even with delisted tickers in the database, engines disagree on the exit price. QuantConnect’s
LEANcloses every delisted position at its last traded price; Rearview closes an acquisition at its last close, and applies a configurable performance-delisting haircut (by default 30% on NYSE and 55% on Nasdaq, following the historical averages reported by Shumway (1997) and Shumway and Warther (1999), though real delisting losses vary widely around those averages). Some tools silently drop a stock whose CSV ends without booking an exit at all.
On mean-reversion strategies that deliberately buy sharp drops, delisted stocks are disproportionately selected as candidates. Leaving them out or closing them at par can turn a losing rule into a paper winner.
4. Cash shortfalls, order sequence and commission-aware sizing
Suppose your portfolio holds 10 positions at 10% of equity each, and your signals are generated at Monday’s close. On Tuesday morning, two buys gap up 4%, leaving your account $380 short of buying the exact share counts computed from Monday’s close.
Backtesters handle that $380 shortfall in at least four different ways:
- Shrink the share count to fit available cash (Rearview’s rule).
- Skip the order entirely because buying power does not cover the requested shares (
LEAN’s default). - Allow cash to go negative on margin or an implicit overdraft (
bt’s default when target weights plus costs exceed cash). - Pre-shrink every order so the commission fits inside the target dollar amount (
bt’s commission-aware rebalancer).
Order sequence matters just as much: whether sells execute before buys at the open, whether stops are live on the entry day itself, and whether a stock that gaps past a stop fills at the open or at the stop price.
5. Tie-breaking and calendar conventions
Two subtle portfolio mechanics surprise even experienced quants:
- Deterministic tie-breaking. Suppose your strategy holds at most 10 stocks, 3 slots are open, and 9 candidates have the exact same integer or rounded rank score (common with discrete filters, bar counts or short-window indicators). One engine breaks ties alphabetically by ticker; another by internal security ID; a third by hash-map iteration order (which can even change between runs). Giving the last slot to stock A instead of stock B changes Tuesday’s cash, which changes Wednesday’s sizing, which cascades across the rest of the decade.
- 252 vs. 365 trading days. If your strategy holds coins (or mixes US stocks with 24/7 crypto), your equity curve has 365 daily returns a year rather than 252. Annualising Sharpe, Sortino and volatility with
sqrt(252)instead ofsqrt(365)understates Sharpe, Sortino and volatility by about 17% (sqrt(252 / 365) ≈ 0.83; the 365-day figure issqrt(365 / 252) ≈ 1.20xlarger). The same issue appears when combining two international exchanges (such as NYSE and ASX) whose combined calendar has more than 252 trading days a year.
A 5-step protocol for reconciling two backtesters
When you move a strategy to a new engine and the headline numbers disagree, do not stare at CAGR or Sharpe. Reconcile from the first fill forward:
- Zero out costs, interest and margin. Set commissions, slippage and cash interest to zero in both tools first.
- Match fill timing and sizing. Align
next_openvs.same_close, whole vs. fractional shares, and confirm whether positions size from signal-day close equity. - Inspect the first diverging trade. Sort the trade lists chronologically and find the very first date where the two engines bought a different ticker, a different share count or filled at a different price.
- Check that single day’s candidates and prices. Verify whether the difference came from price adjustment (split or dividend), universe membership, a tie in rank, or cash remaining after an overnight gap.
- Re-enable costs and corporate actions one at a time. Once zero-cost trades match, turn commissions, slippage, margin and delisting haircuts back on so you know exactly which rule accounts for any remaining gap.
For Rearview’s exact defaults and how to match AmiBroker, RealTest or Wealth-Lab step by step, see Why your results differ from another backtester and How Rearview is checked.