Rearview Download

Delta-debugging trading strategies: finding which rules earn their place, and the least friction that breaks them

Most systematic trading strategies do not start complicated. They start with one clean idea, such as buying a short-term dip in an uptrend or buying stocks near a 52-week high, and then accumulate guardrails over weeks of backtesting: a trend filter to dodge one bad quarter, a secondary volatility gate, a second exit clause, a holding-period cap.

Each extra condition looks sensible on the equity curve that inspired it. Yet once a strategy has five or six conditions stacked together, two uncomfortable questions are surprisingly hard to answer:

  1. How many of those conditions actually earn their place beyond the statistical noise of the period, once interactions between rules are accounted for?
  2. How little real-world friction does it take to break the strategy? Not at one arbitrary stress level like "double costs", but the exact extra slippage, bar delay, later start, missed best trades, missed top securities, or delisting wipeout at which it stops beating idle cash.

In Rearview 1.1, we implemented both checks as a single automated analysis (What matters, rule C115). The first half borrows delta debugging (ddmin) from compiler and bug-minimization research and pairs it with a paired stationary block bootstrap as the statistical oracle. The second half searches six execution and concentration frictions using integer bisection, linear scans, a second ddmin pass over the strategy’s top closed trades, and a joint proportional friction sweep.

This note walks through the mechanics, why naive leave-one-out ablation misses rule interactions, why pruning rules must still count against your multiple-testing budget, and the honest limits of both algorithms.

1. Why naive leave-one-out ablation misleads you

The obvious way to test whether a rule matters is leave-one-out ablation: remove one condition at a time, rerun the backtest, and check whether the Sharpe ratio fell.

That runs into two problems immediately:

  1. Raw Sharpe differences ignore sampling noise. Over ten years of daily returns, two variants of a strategy can differ in annualised Sharpe by 0.15 or 0.25 purely because of a handful of volatile weeks. Looking at the point estimate alone invites you to keep every filter that nudges historical Sharpe upward by a hair.
  2. Conditions interact. Two filters can overlap, or an entry filter and an exit clause can cancel each other’s side effects. Removing condition A alone can hurt the Sharpe ratio substantially, yet removing A together with B and C can land right back where the full strategy started.

Rearview’s reference example rsi2_pullback on the zero-cost synthetic 10-year sample dataset (500 securities: stocks, a market ETF and a volatility index, not a real market) illustrates this cleanly:

enter long when regime and close > sma(close, 200) and rsi2 < rsi_entry
exit when  rsi2 > 70 or bars_held >= 10

That script has five removable clauses: three top-level and operands in enter long when (regime, close > sma(close, 200), rsi2 < rsi_entry) and two top-level or operands in exit when (rsi2 > 70, bars_held >= 10).

Removing close > sma(close, 200) alone drops the strategy’s Sharpe ratio by 0.63. A trader doing single-condition ablation would immediately conclude that the 200-day moving average filter is load-bearing. Yet when delta debugging removes it alongside the other three clauses (rsi2 < rsi_entry, rsi2 > 70, and bars_held >= 10), the stripped strategy’s Sharpe ratio lands right back inside the noise of the period, keeping only 1 of the 5 conditions: the market trend filter regime. On synthetic prices that contain no real short-term pullback pattern, the only component doing real work was staying long equities during regime uptrends.

2. Turning a strategy into removable terms

To automate rule reduction cleanly without rewriting arbitrary code, the parser extracts removable operands directly from the strategy’s abstract syntax tree (AST), whether for a single script or pooled across every leg of a multi-strategy portfolio:

  • Entry clauses: an enter … when C statement whose condition is a top-level chain of and yields each operand as a term. A parenthesised expression stays one atomic operand, an entry with a single operand has no removable terms, and any candidate subset that would leave an entry with zero conditions is rejected as invalid.
  • Exit clauses: an exit when C statement yields the operands of its top-level or chain (including a single operand as one term). Removing every operand of an exit drops that exit when statement while leaving stops, trailing stops, targets or other exits intact.

Because every term carries its exact byte span (start, end, line, col) from the lexer, the Mac app can strike through the redundant conditions directly inside the strategy editor and print the minimal surviving script via the AST printer, guaranteed to parse back to the exact same tree.

3. The oracle: paired stationary block bootstrap

To decide whether a smaller variant is statistically distinguishable from the original run, you cannot compare their Sharpes with an independent two-sample test. Both runs traded the same market on the same dates, so their daily returns are strongly correlated; ignoring that correlation makes the confidence interval far too wide and masks genuine differences.

Instead, Rearview uses the same paired history interval it uses when comparing any two runs on the same dataset (rule C112):

  1. Compute time-weighted daily returns for both the original run A and the reduced variant B, and align them on their common trading dates (n >= 20 returns required).
  2. Draw 2,000 stationary block-bootstrap resamples (Politis & Romano, 1994) with a geometric block length averaging 20 trading days (p = 1/20, roughly one trading month). This is Rearview’s shared bootstrap block length, preserving short-term autocorrelation and volatility clustering while leaving enough independent blocks across a multi-year history. Crucially, each resample k draws one shared sequence of day indices and evaluates both A and B on those exact same days.
  3. On each resample k, compute the annualised Sharpe difference ΔSR_k = SR_ann(B_k) - SR_ann(A_k).
  4. Take the type-7 2.5% and 97.5% quantiles of the defined differences to form a two-sided 95% percentile confidence interval [lo, hi].

If lo <= 0 <= hi, the variant is inside period noise: on this history, removing those conditions produces a Sharpe ratio (slightly lower or slightly higher) that cannot be separated from the original run’s. Resampling is deterministically seeded (Xoshiro256, fixed seed 1), so the exact same run always produces the exact same interval.

4. Shrinking the strategy with Zeller’s ddmin

Before running delta debugging, Rearview runs the leave-one-out sweep (alone) over every term. That serves two purposes: the UI reports each individual condition’s Sharpe delta and 95% interval so you can contrast solo impact against combined pruning, and every variant evaluated during alone pre-populates the memoisation cache (by_removal) so ddmin never reruns the same removal set twice.

With the boolean oracle inside(removed) in place, finding a minimal kept subset of conditions K (whose complement L \ K can be removed while staying inside noise) maps straight onto Andreas Zeller and Ralf Hildebrandt’s delta debugging (ddmin) algorithm (2002), originally designed to shrink failing compiler inputs to a minimal test case.

Starting from K = L (the full set of terms, where removing nothing trivially differs by 0), ddmin partitions K into n = 2 chunks and iterates:

  1. Test keeping each chunk alone (dropping everything outside that chunk to try aggressive reductions first). If inside(L \ chunk) holds, shrink K to that chunk and reset n = 2.
  2. If n > 2, test keeping each complement (dropping one chunk of K at a time). If inside(L \ complement) holds, shrink K to that complement and set n = max(n - 1, 2).
  3. Increase granularity to n = min(2n, |K|) and repeat until n >= |K| (plus a final empty-set check if |K| = 1, which succeeds only if the single surviving term is an exit clause).

Counting ablation variants in the evidence budget

There is a subtle trap with automated strategy pruning: if you run Find What Matters, inspect ten simplified scripts, and adopt one whose Sharpe looks cleaner, you have just performed another form of multiple testing.

Rearview avoids turning pruning into hidden p-hacking by treating every script variant evaluated during alone and ddmin as a real try on the dataset. For each variant, the engine computes a token-based try key (the first 32 hex digits of SHA-256 over {"code", "params", "run"}, where code hashes the printed variant’s lexed token stream without comments or whitespace and run omits the benchmark) and writes it into shrink.json. The Mac app automatically records those variants into the project’s SHA-256-chained tries.jsonl log (kind: "shrink"), and CLI users record them with rvw tries add <log> --as <path> --data <entry> --shrink --folder <run-folder>. Those variants count toward the project’s evidence budget and Deflated Sharpe Ratio (while remaining excluded from the Forward Ledger). By contrast, the friction runs in the second half keep the script fixed and perturb only execution assumptions, so they do not spend research tries.

5. The smallest break: searching six failure modes (plus ddmin on trades)

Knowing which rules you can delete is half the picture. The other half is knowing how thin your margin of safety is against real-world execution and concentration risk.

Conventional stress tests check one fixed point, such as doubling commissions or delaying signals by one bar, and leave you guessing whether the strategy would have collapsed at 1.2x friction or held out to 4x. Rearview’s smallest break inverts the question: what is the smallest friction at which the strategy’s total return falls to or below idle cash (under the run’s own cash-interest settings over the same window)?

Beating idle cash is deliberately an assumption-free necessary floor, not a sufficient live-trading bar: it avoids guessing a benchmark or risk-free rate, and works identically for long-only, long/short, crypto or futures strategies. For the slippage, later-start and delistings breaks, the Mac app puts a one-click button beside the break to open that exact breaking configuration as an ordinary run so you can inspect its Sharpe ratio, drawdown and trade list directly. (If the original run already earns no more than idle cash, Rearview flags it as already broken and skips the friction searches.)

Rather than adjusting P&L after the fact in a spreadsheet, every probe re-simulates the entire portfolio through the engine so that freed cash, skipped entries, and slot contention reallocate realistically across the remaining candidates. Six frictions are searched within fixed bounds:

FrictionSearch boundSearch methodWhat it actually does to the simulation
Extra slippage1..=50 bpInteger bisectionAdds x basis points on top of the run’s configured slippage (even if the run had 0 slippage)
Signals late1..=2 barsLinear scanShifts every signal to its security’s next bar, x times
Later start1..=⌊D/3⌋ daysInteger bisectionMoves the start date forward by x daily records and compares against idle cash from that same later start date
Best trades missedTop max(1, ⌊closed / 50⌋) (2%) tradesWhole-set check, then ddminRemoves the actual filled entry signal rows of the top-P&L closed trades; if dropping the top 2% breaks the run, ddmin finds a 1-minimal subset of those trades whose absence breaks it
Top securities missedTop 1..=3 securitiesLinear scanRemoves every entry signal for the top x securities ranked by summed closed-trade P&L
Performance delistingsOn / OffSingle checkForces 100% total loss on every performance delisting in the run’s date range (instead of the exchange’s default haircut)

Two engineering details matter here:

  • Re-simulation beats post-hoc trade subtraction. If you delete a strategy’s three luckiest trades in a spreadsheet, you assume the cash sat idle. In Rearview, deleting those trades removes their entry signal rows on the days those entry orders were created and reruns the engine, so the next-ranked stock takes the slot instead. Sometimes the replacement trade saves the portfolio; sometimes missing two lottery tickets leaves the whole strategy underwater.
  • Joint proportional friction (together). A strategy can survive every single stress test in isolation (holding up to 20 bp slippage alone, or 1 bar late alone, or missing its best trade alone) yet collapse as soon as several mild frictions arrive at once. After testing each friction alone, Rearview bisects a shared fraction f = j / 64 (j ∈ 1..=64), applying every friction simultaneously at floor(bound × f) (with performance delistings switching to total loss at f >= 0.5).

On the zero-cost synthetic sample data:

  • rsi2_pullback breaks at 8 bp of slippage alone (which is why adding realistic costs of $0.005/share and 5 bp slippage already pushes that synthetic run below cash). Under together, it breaks at 14% (j = 9/64), where integer flooring applies 7 bp slippage alongside a shifted start and the top ~14% of its 2% best-trade bound (late and securities still floor to 0 and delistings is off), breaking just below its solo slippage limit.
  • near_52w_high shrinks to 1 of 3 conditions and still holds above idle cash against every friction bound, even all of them at once.

6. Performance and differential verification

Automated ablation and friction searches are only useful if they are fast enough to run routinely after an ordinary backtest. Because each friction probe reuses the pre-evaluated signal matrices and runs the compiled Rust engine directly in memory, our shrink_500_seconds release benchmark (running rvw shrink on rsi2_pullback over 500 securities × 10 years (2,520 days) with commission and slippage on the benchmark synthetic dataset) finishes in 1.70 seconds (1.698s measured for Rearview 1.1.0 on an Apple M5 Max, 4 threads; see How Rearview is checked).

Like the rest of the engine, rule C115 is cross-checked against our clean-room Python reference implementation (reference/refbt/shrink.py) before every release. On Rearview 1.1.0, 38 of 38 scenarios matched between the Rust CLI and the reference implementation with zero differences, including multi-strategy portfolios, where writing the cross-check caught two subtle edge cases early (per-sleeve trade-ID numbering collisions when ranking closed trades across sleeves, and zero-bound bisections).

7. Honest limits

Delta debugging and friction searches are sharp diagnostic tools, not magic proofs:

  • ddmin guarantees 1-minimality, not the global minimum. Because a trading strategy’s Sharpe difference is not strictly monotone as conditions are removed (nor is breaking strictly monotone as trades are removed), ddmin stops at a subset where no further chunk or single-item removal passes the oracle. When two different minimal subsets exist, the one returned depends on order.
  • Syntactic granularity. Term extraction splits top-level and chains in entries and or chains in exits. A compound expression hidden behind a variable binding (such as regime or a parenthesised sub-expression) is tested as one unit unless written out in the entry or exit clause.
  • "Inside noise" is two-sided, and is not the same as "useless". The paired bootstrap uses a percentile 95% interval (rather than a studentized Ledoit-Wolf test) and tests lo <= 0 <= hi symmetrically. On a short history or a high-volatility strategy, the interval can be wide enough that a genuinely sensible risk filter lands inside noise simply for lack of statistical power, or a pruned variant that happens to score slightly higher still sits inside noise. Treat a struck-through condition as a prompt to ask "what evidence do I actually have for this clause?", not an automatic delete button.
  • Bisection assumes rough monotonicity. Rarely, higher slippage or a later start can alter cash availability enough to accidentally skip a large losing trade, causing total return to dip below idle cash at one level and bounce back slightly at a higher level. Bisection finds a small breaking level in O(log B) runs, not an exhaustive scan of every integer.
  • Historical robustness cannot test unseen regimes. Surviving 50 bp of slippage and missing your top three stocks over the last decade says the historical edge was not concentrated in a handful of lucky fills. It cannot guarantee the underlying market effect will persist tomorrow.

Used alongside Rearview 1.1’s other checks, including 200 automatic execution copies after every run (Copies), cross-script trial accounting (The evidence budget), and a sealed lockbox, Find What Matters turns two vague intuitions ("this script feels over-engineered" and "this edge feels fragile") into concrete, reproducible numbers in under two seconds.

Read the full specification in What matters, or test how sensitive your trial count already is in the client-side Deflated Sharpe Ratio calculator.