<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:base="https://rearview.page/writing/">
<title>Writing · Rearview</title><id>https://rearview.page/writing/</id>
<link rel="self" href="https://rearview.page/writing/feed.xml"/><link rel="alternate" type="text/html" href="https://rearview.page/writing/"/>
<updated>2026-11-20T00:00:00Z</updated><author><name>Rearview</name></author>
<entry><title>Delta-debugging trading strategies: finding which rules earn their place</title><id>https://rearview.page/writing/delta-debugging-trading-strategies/</id><link rel="alternate" type="text/html" href="https://rearview.page/writing/delta-debugging-trading-strategies/"/><published>2026-11-20T00:00:00Z</published><updated>2026-11-20T00:00:00Z</updated><summary>Using Zeller's ddmin algorithm with a paired stationary block bootstrap to strip noisy strategy conditions, plus searching six real-world frictions to find the smallest break.</summary><content type="html">&lt;p&gt;Most systematic trading strategies do not start complicated. They start with one clean idea, such as buying a short-term dip in an uptrend or buying stocks near a 52-week high, and then accumulate guardrails over weeks of backtesting: a trend filter to dodge one bad quarter, a secondary volatility gate, a second exit clause, a holding-period cap.&lt;/p&gt;&lt;p&gt;Each extra condition looks sensible on the equity curve that inspired it. Yet once a strategy has five or six conditions stacked together, two uncomfortable questions are surprisingly hard to answer:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;How many of those conditions actually earn their place&lt;/b&gt; beyond the statistical noise of the period, once interactions between rules are accounted for?&lt;/li&gt;&lt;li&gt;&lt;b&gt;How little real-world friction does it take to break the strategy?&lt;/b&gt; Not at one arbitrary stress level like &amp;quot;double costs&amp;quot;, but the exact extra slippage, bar delay, later start, missed best trades, missed top securities, or delisting wipeout at which it stops beating idle cash.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;In Rearview 1.1, we implemented both checks as a single automated analysis (&lt;b&gt;What matters&lt;/b&gt;, rule C115). The first half borrows &lt;b&gt;delta debugging (&lt;code&gt;ddmin&lt;/code&gt;)&lt;/b&gt; from compiler and bug-minimization research and pairs it with a &lt;b&gt;paired stationary block bootstrap&lt;/b&gt; as the statistical oracle. The second half searches six execution and concentration frictions using integer bisection, linear scans, a second &lt;b&gt;&lt;code&gt;ddmin&lt;/code&gt; pass over the strategy’s top closed trades&lt;/b&gt;, and a &lt;b&gt;joint proportional friction sweep&lt;/b&gt;.&lt;/p&gt;&lt;p&gt;This note walks through the mechanics, why naive leave-one-out ablation misses rule interactions, why pruning rules must still count against your multiple-testing budget, and the honest limits of both algorithms.&lt;/p&gt;&lt;h2 id=&quot;1-why-naive-leave-one-out-ablation-misleads-you&quot;&gt;1. Why naive leave-one-out ablation misleads you&lt;/h2&gt;&lt;p&gt;The obvious way to test whether a rule matters is leave-one-out ablation: remove one condition at a time, rerun the backtest, and check whether the Sharpe ratio fell.&lt;/p&gt;&lt;p&gt;That runs into two problems immediately:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Raw Sharpe differences ignore sampling noise.&lt;/b&gt; Over ten years of daily returns, two variants of a strategy can differ in annualised Sharpe by &lt;code&gt;0.15&lt;/code&gt; or &lt;code&gt;0.25&lt;/code&gt; purely because of a handful of volatile weeks. Looking at the point estimate alone invites you to keep every filter that nudges historical Sharpe upward by a hair.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Conditions interact.&lt;/b&gt; Two filters can overlap, or an entry filter and an exit clause can cancel each other’s side effects. Removing condition A alone can hurt the Sharpe ratio substantially, yet removing A together with B and C can land right back where the full strategy started.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;Rearview’s reference example &lt;code&gt;rsi2_pullback&lt;/code&gt; on the zero-cost synthetic 10-year sample dataset (500 securities: stocks, a market ETF and a volatility index, not a real market) illustrates this cleanly:&lt;/p&gt;&lt;div class=&quot;code&quot;&gt;&lt;pre&gt;&lt;code&gt;enter long when regime and close &amp;gt; sma(close, 200) and rsi2 &amp;lt; rsi_entry
exit when  rsi2 &amp;gt; 70 or bars_held &amp;gt;= 10&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;That script has five removable clauses: three top-level &lt;code&gt;and&lt;/code&gt; operands in &lt;code&gt;enter long when&lt;/code&gt; (&lt;code&gt;regime&lt;/code&gt;, &lt;code&gt;close &amp;gt; sma(close, 200)&lt;/code&gt;, &lt;code&gt;rsi2 &amp;lt; rsi_entry&lt;/code&gt;) and two top-level &lt;code&gt;or&lt;/code&gt; operands in &lt;code&gt;exit when&lt;/code&gt; (&lt;code&gt;rsi2 &amp;gt; 70&lt;/code&gt;, &lt;code&gt;bars_held &amp;gt;= 10&lt;/code&gt;).&lt;/p&gt;&lt;p&gt;Removing &lt;code&gt;close &amp;gt; sma(close, 200)&lt;/code&gt; &lt;b&gt;alone&lt;/b&gt; drops the strategy’s Sharpe ratio by &lt;b&gt;0.63&lt;/b&gt;. A trader doing single-condition ablation would immediately conclude that the 200-day moving average filter is load-bearing. Yet when delta debugging removes it alongside the other three clauses (&lt;code&gt;rsi2 &amp;lt; rsi_entry&lt;/code&gt;, &lt;code&gt;rsi2 &amp;gt; 70&lt;/code&gt;, and &lt;code&gt;bars_held &amp;gt;= 10&lt;/code&gt;), the stripped strategy’s Sharpe ratio lands right back inside the noise of the period, keeping only &lt;b&gt;1 of the 5 conditions&lt;/b&gt;: the market trend filter &lt;code&gt;regime&lt;/code&gt;. On synthetic prices that contain no real short-term pullback pattern, the only component doing real work was staying long equities during &lt;code&gt;regime&lt;/code&gt; uptrends.&lt;/p&gt;&lt;h2 id=&quot;2-turning-a-strategy-into-removable-terms&quot;&gt;2. Turning a strategy into removable terms&lt;/h2&gt;&lt;p&gt;To automate rule reduction cleanly without rewriting arbitrary code, the parser extracts removable operands directly from the strategy’s abstract syntax tree (AST), whether for a single script or pooled across every leg of a multi-strategy portfolio:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Entry clauses:&lt;/b&gt; an &lt;code&gt;enter … when C&lt;/code&gt; statement whose condition is a top-level chain of &lt;code&gt;and&lt;/code&gt; yields each operand as a term. A parenthesised expression stays one atomic operand, an entry with a single operand has no removable terms, and any candidate subset that would leave an entry with zero conditions is rejected as invalid.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Exit clauses:&lt;/b&gt; an &lt;code&gt;exit when C&lt;/code&gt; statement yields the operands of its top-level &lt;code&gt;or&lt;/code&gt; chain (including a single operand as one term). Removing every operand of an exit drops that &lt;code&gt;exit when&lt;/code&gt; statement while leaving stops, trailing stops, targets or other exits intact.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Because every term carries its exact byte span (&lt;code&gt;start&lt;/code&gt;, &lt;code&gt;end&lt;/code&gt;, &lt;code&gt;line&lt;/code&gt;, &lt;code&gt;col&lt;/code&gt;) from the lexer, the Mac app can strike through the redundant conditions directly inside the strategy editor and print the minimal surviving script via the AST printer, guaranteed to parse back to the exact same tree.&lt;/p&gt;&lt;h2 id=&quot;3-the-oracle-paired-stationary-block-bootstrap&quot;&gt;3. The oracle: paired stationary block bootstrap&lt;/h2&gt;&lt;p&gt;To decide whether a smaller variant is statistically distinguishable from the original run, you cannot compare their Sharpes with an independent two-sample test. Both runs traded the same market on the same dates, so their daily returns are strongly correlated; ignoring that correlation makes the confidence interval far too wide and masks genuine differences.&lt;/p&gt;&lt;p&gt;Instead, Rearview uses the same &lt;b&gt;paired history interval&lt;/b&gt; it uses when comparing any two runs on the same dataset (rule C112):&lt;/p&gt;&lt;ol&gt;&lt;li&gt;Compute time-weighted daily returns for both the original run &lt;code&gt;A&lt;/code&gt; and the reduced variant &lt;code&gt;B&lt;/code&gt;, and align them on their common trading dates (&lt;code&gt;n &amp;gt;= 20&lt;/code&gt; returns required).&lt;/li&gt;&lt;li&gt;Draw &lt;b&gt;2,000 stationary block-bootstrap resamples&lt;/b&gt; (Politis &amp;amp; Romano, 1994) with a geometric block length averaging &lt;b&gt;20 trading days&lt;/b&gt; (&lt;code&gt;p = 1/20&lt;/code&gt;, roughly one trading month). This is Rearview’s shared bootstrap block length, preserving short-term autocorrelation and volatility clustering while leaving enough independent blocks across a multi-year history. Crucially, each resample &lt;code&gt;k&lt;/code&gt; draws &lt;b&gt;one shared sequence of day indices&lt;/b&gt; and evaluates both &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; on those exact same days.&lt;/li&gt;&lt;li&gt;On each resample &lt;code&gt;k&lt;/code&gt;, compute the annualised Sharpe difference &lt;code&gt;ΔSR_k = SR_ann(B_k) - SR_ann(A_k)&lt;/code&gt;.&lt;/li&gt;&lt;li&gt;Take the type-7 &lt;b&gt;2.5% and 97.5% quantiles&lt;/b&gt; of the defined differences to form a two-sided 95% percentile confidence interval &lt;code&gt;[lo, hi]&lt;/code&gt;.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;If &lt;code&gt;lo &amp;lt;= 0 &amp;lt;= hi&lt;/code&gt;, the variant is &lt;b&gt;inside period noise&lt;/b&gt;: on this history, removing those conditions produces a Sharpe ratio (slightly lower or slightly higher) that cannot be separated from the original run’s. Resampling is deterministically seeded (&lt;code&gt;Xoshiro256&lt;/code&gt;, fixed seed &lt;code&gt;1&lt;/code&gt;), so the exact same run always produces the exact same interval.&lt;/p&gt;&lt;h2 id=&quot;4-shrinking-the-strategy-with-zellers-ddmin&quot;&gt;4. Shrinking the strategy with Zeller’s &lt;code&gt;ddmin&lt;/code&gt;&lt;/h2&gt;&lt;p&gt;Before running delta debugging, Rearview runs the leave-one-out sweep (&lt;code&gt;alone&lt;/code&gt;) over every term. That serves two purposes: the UI reports each individual condition’s Sharpe delta and 95% interval so you can contrast solo impact against combined pruning, and every variant evaluated during &lt;code&gt;alone&lt;/code&gt; pre-populates the memoisation cache (&lt;code&gt;by_removal&lt;/code&gt;) so &lt;code&gt;ddmin&lt;/code&gt; never reruns the same removal set twice.&lt;/p&gt;&lt;p&gt;With the boolean oracle &lt;code&gt;inside(removed)&lt;/code&gt; in place, finding a minimal &lt;b&gt;kept&lt;/b&gt; subset of conditions &lt;code&gt;K&lt;/code&gt; (whose complement &lt;code&gt;L \ K&lt;/code&gt; can be removed while staying inside noise) maps straight onto Andreas Zeller and Ralf Hildebrandt’s &lt;b&gt;delta debugging (&lt;code&gt;ddmin&lt;/code&gt;)&lt;/b&gt; algorithm (2002), originally designed to shrink failing compiler inputs to a minimal test case.&lt;/p&gt;&lt;p&gt;Starting from &lt;code&gt;K = L&lt;/code&gt; (the full set of terms, where removing nothing trivially differs by &lt;code&gt;0&lt;/code&gt;), &lt;code&gt;ddmin&lt;/code&gt; partitions &lt;code&gt;K&lt;/code&gt; into &lt;code&gt;n = 2&lt;/code&gt; chunks and iterates:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Test keeping each chunk alone&lt;/b&gt; (dropping everything outside that chunk to try aggressive reductions first). If &lt;code&gt;inside(L \ chunk)&lt;/code&gt; holds, shrink &lt;code&gt;K&lt;/code&gt; to that chunk and reset &lt;code&gt;n = 2&lt;/code&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;If &lt;code&gt;n &amp;gt; 2&lt;/code&gt;, test keeping each complement&lt;/b&gt; (dropping one chunk of &lt;code&gt;K&lt;/code&gt; at a time). If &lt;code&gt;inside(L \ complement)&lt;/code&gt; holds, shrink &lt;code&gt;K&lt;/code&gt; to that complement and set &lt;code&gt;n = max(n - 1, 2)&lt;/code&gt;.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Increase granularity&lt;/b&gt; to &lt;code&gt;n = min(2n, |K|)&lt;/code&gt; and repeat until &lt;code&gt;n &amp;gt;= |K|&lt;/code&gt; (plus a final empty-set check if &lt;code&gt;|K| = 1&lt;/code&gt;, which succeeds only if the single surviving term is an exit clause).&lt;/li&gt;&lt;/ol&gt;&lt;h3 id=&quot;counting-ablation-variants-in-the-evidence-budget&quot;&gt;Counting ablation variants in the evidence budget&lt;/h3&gt;&lt;p&gt;There is a subtle trap with automated strategy pruning: if you run &lt;code&gt;Find What Matters&lt;/code&gt;, inspect ten simplified scripts, and adopt one whose Sharpe looks cleaner, you have just performed another form of &lt;b&gt;multiple testing&lt;/b&gt;.&lt;/p&gt;&lt;p&gt;Rearview avoids turning pruning into hidden p-hacking by treating every script variant evaluated during &lt;code&gt;alone&lt;/code&gt; and &lt;code&gt;ddmin&lt;/code&gt; as a real try on the dataset. For each variant, the engine computes a token-based try key (the first 32 hex digits of SHA-256 over &lt;code&gt;{&amp;quot;code&amp;quot;, &amp;quot;params&amp;quot;, &amp;quot;run&amp;quot;}&lt;/code&gt;, where &lt;code&gt;code&lt;/code&gt; hashes the printed variant’s lexed token stream without comments or whitespace and &lt;code&gt;run&lt;/code&gt; omits the benchmark) and writes it into &lt;code&gt;shrink.json&lt;/code&gt;. The Mac app automatically records those variants into the project’s SHA-256-chained &lt;code&gt;tries.jsonl&lt;/code&gt; log (&lt;code&gt;kind: &amp;quot;shrink&amp;quot;&lt;/code&gt;), and CLI users record them with &lt;code&gt;rvw tries add &amp;lt;log&amp;gt; --as &amp;lt;path&amp;gt; --data &amp;lt;entry&amp;gt; --shrink --folder &amp;lt;run-folder&amp;gt;&lt;/code&gt;. Those variants count toward the project’s &lt;b&gt;evidence budget&lt;/b&gt; and &lt;b&gt;Deflated Sharpe Ratio&lt;/b&gt; (while remaining excluded from the Forward Ledger). By contrast, the friction runs in the second half keep the script fixed and perturb only execution assumptions, so they do not spend research tries.&lt;/p&gt;&lt;h2 id=&quot;5-the-smallest-break-searching-six-failure-modes-plus-ddmin-on-trades&quot;&gt;5. The smallest break: searching six failure modes (plus &lt;code&gt;ddmin&lt;/code&gt; on trades)&lt;/h2&gt;&lt;p&gt;Knowing which rules you can delete is half the picture. The other half is knowing how thin your margin of safety is against real-world execution and concentration risk.&lt;/p&gt;&lt;p&gt;Conventional stress tests check one fixed point, such as doubling commissions or delaying signals by one bar, and leave you guessing whether the strategy would have collapsed at 1.2x friction or held out to 4x. Rearview’s &lt;b&gt;smallest break&lt;/b&gt; inverts the question: &lt;b&gt;what is the smallest friction at which the strategy’s total return falls to or below idle cash&lt;/b&gt; (under the run’s own cash-interest settings over the same window)?&lt;/p&gt;&lt;p&gt;Beating idle cash is deliberately an assumption-free &lt;b&gt;necessary floor&lt;/b&gt;, not a sufficient live-trading bar: it avoids guessing a benchmark or risk-free rate, and works identically for long-only, long/short, crypto or futures strategies. For the slippage, later-start and delistings breaks, the Mac app puts a one-click button beside the break to open that exact breaking configuration as an ordinary run so you can inspect its Sharpe ratio, drawdown and trade list directly. (If the original run already earns no more than idle cash, Rearview flags it as already broken and skips the friction searches.)&lt;/p&gt;&lt;p&gt;Rather than adjusting P&amp;amp;L after the fact in a spreadsheet, every probe &lt;b&gt;re-simulates the entire portfolio&lt;/b&gt; through the engine so that freed cash, skipped entries, and slot contention reallocate realistically across the remaining candidates. Six frictions are searched within fixed bounds:&lt;/p&gt;&lt;div class=&quot;table&quot;&gt;&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Friction&lt;/th&gt;&lt;th&gt;Search bound&lt;/th&gt;&lt;th&gt;Search method&lt;/th&gt;&lt;th&gt;What it actually does to the simulation&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Extra slippage&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&lt;code&gt;1..=50 bp&lt;/code&gt;&lt;/td&gt;&lt;td&gt;Integer bisection&lt;/td&gt;&lt;td&gt;Adds &lt;code&gt;x&lt;/code&gt; basis points on top of the run’s configured slippage (even if the run had &lt;code&gt;0&lt;/code&gt; slippage)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Signals late&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&lt;code&gt;1..=2&lt;/code&gt; bars&lt;/td&gt;&lt;td&gt;Linear scan&lt;/td&gt;&lt;td&gt;Shifts every signal to its security’s next bar, &lt;code&gt;x&lt;/code&gt; times&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Later start&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&lt;code&gt;1..=⌊D/3⌋&lt;/code&gt; days&lt;/td&gt;&lt;td&gt;Integer bisection&lt;/td&gt;&lt;td&gt;Moves the start date forward by &lt;code&gt;x&lt;/code&gt; daily records and compares against idle cash from that same later start date&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Best trades missed&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Top &lt;code&gt;max(1, ⌊closed / 50⌋)&lt;/code&gt; (2%) trades&lt;/td&gt;&lt;td&gt;Whole-set check, then &lt;b&gt;&lt;code&gt;ddmin&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Removes the actual filled entry signal rows of the top-P&amp;amp;L closed trades; if dropping the top 2% breaks the run, &lt;code&gt;ddmin&lt;/code&gt; finds a &lt;b&gt;1-minimal subset&lt;/b&gt; of those trades whose absence breaks it&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Top securities missed&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Top &lt;code&gt;1..=3&lt;/code&gt; securities&lt;/td&gt;&lt;td&gt;Linear scan&lt;/td&gt;&lt;td&gt;Removes every entry signal for the top &lt;code&gt;x&lt;/code&gt; securities ranked by summed closed-trade P&amp;amp;L&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;Performance delistings&lt;/b&gt;&lt;/td&gt;&lt;td&gt;On / Off&lt;/td&gt;&lt;td&gt;Single check&lt;/td&gt;&lt;td&gt;Forces 100% total loss on every performance delisting in the run’s date range (instead of the exchange’s default haircut)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;p&gt;Two engineering details matter here:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Re-simulation beats post-hoc trade subtraction.&lt;/b&gt; If you delete a strategy’s three luckiest trades in a spreadsheet, you assume the cash sat idle. In Rearview, deleting those trades removes their &lt;b&gt;entry signal rows&lt;/b&gt; on the days those entry orders were created and reruns the engine, so the next-ranked stock takes the slot instead. Sometimes the replacement trade saves the portfolio; sometimes missing two lottery tickets leaves the whole strategy underwater.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Joint proportional friction (&lt;code&gt;together&lt;/code&gt;).&lt;/b&gt; A strategy can survive every single stress test in isolation (holding up to 20 bp slippage alone, or 1 bar late alone, or missing its best trade alone) yet collapse as soon as several mild frictions arrive at once. After testing each friction alone, Rearview bisects a shared fraction &lt;code&gt;f = j / 64&lt;/code&gt; (&lt;code&gt;j ∈ 1..=64&lt;/code&gt;), applying every friction simultaneously at &lt;code&gt;floor(bound × f)&lt;/code&gt; (with performance delistings switching to total loss at &lt;code&gt;f &amp;gt;= 0.5&lt;/code&gt;).&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;On the zero-cost synthetic sample data:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;code&gt;rsi2_pullback&lt;/code&gt; breaks at &lt;b&gt;8 bp of slippage&lt;/b&gt; alone (which is why adding realistic costs of &lt;code&gt;$0.005/share&lt;/code&gt; and &lt;code&gt;5 bp&lt;/code&gt; slippage already pushes that synthetic run below cash). Under &lt;code&gt;together&lt;/code&gt;, it breaks at &lt;b&gt;14%&lt;/b&gt; (&lt;code&gt;j = 9/64&lt;/code&gt;), where integer flooring applies &lt;code&gt;7 bp&lt;/code&gt; slippage alongside a shifted start and the top ~14% of its 2% best-trade bound (&lt;code&gt;late&lt;/code&gt; and &lt;code&gt;securities&lt;/code&gt; still floor to &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;delistings&lt;/code&gt; is off), breaking just below its solo slippage limit.&lt;/li&gt;&lt;li&gt;&lt;code&gt;near_52w_high&lt;/code&gt; shrinks to &lt;b&gt;1 of 3 conditions&lt;/b&gt; and still holds above idle cash against &lt;b&gt;every friction bound, even all of them at once&lt;/b&gt;.&lt;/li&gt;&lt;/ul&gt;&lt;h2 id=&quot;6-performance-and-differential-verification&quot;&gt;6. Performance and differential verification&lt;/h2&gt;&lt;p&gt;Automated ablation and friction searches are only useful if they are fast enough to run routinely after an ordinary backtest. Because each friction probe reuses the pre-evaluated signal matrices and runs the compiled Rust engine directly in memory, our &lt;code&gt;shrink_500_seconds&lt;/code&gt; release benchmark (running &lt;code&gt;rvw shrink&lt;/code&gt; on &lt;code&gt;rsi2_pullback&lt;/code&gt; over &lt;b&gt;500 securities × 10 years (2,520 days)&lt;/b&gt; with commission and slippage on the benchmark synthetic dataset) finishes in &lt;b&gt;1.70 seconds&lt;/b&gt; (&lt;code&gt;1.698s&lt;/code&gt; measured for Rearview 1.1.0 on an Apple M5 Max, 4 threads; see &lt;a href=&quot;/docs/verification/#speed&quot;&gt;How Rearview is checked&lt;/a&gt;).&lt;/p&gt;&lt;p&gt;Like the rest of the engine, rule C115 is cross-checked against our clean-room Python reference implementation (&lt;code&gt;reference/refbt/shrink.py&lt;/code&gt;) before every release. On Rearview 1.1.0, &lt;b&gt;38 of 38 scenarios&lt;/b&gt; matched between the Rust CLI and the reference implementation with zero differences, including multi-strategy portfolios, where writing the cross-check caught two subtle edge cases early (per-sleeve trade-ID numbering collisions when ranking closed trades across sleeves, and zero-bound bisections).&lt;/p&gt;&lt;h2 id=&quot;7-honest-limits&quot;&gt;7. Honest limits&lt;/h2&gt;&lt;p&gt;Delta debugging and friction searches are sharp diagnostic tools, not magic proofs:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;&lt;code&gt;ddmin&lt;/code&gt; guarantees 1-minimality, not the global minimum.&lt;/b&gt; Because a trading strategy’s Sharpe difference is not strictly monotone as conditions are removed (nor is breaking strictly monotone as trades are removed), &lt;code&gt;ddmin&lt;/code&gt; stops at a subset where no further chunk or single-item removal passes the oracle. When two different minimal subsets exist, the one returned depends on order.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Syntactic granularity.&lt;/b&gt; Term extraction splits top-level &lt;code&gt;and&lt;/code&gt; chains in entries and &lt;code&gt;or&lt;/code&gt; chains in exits. A compound expression hidden behind a variable binding (such as &lt;code&gt;regime&lt;/code&gt; or a parenthesised sub-expression) is tested as one unit unless written out in the entry or exit clause.&lt;/li&gt;&lt;li&gt;&lt;b&gt;&amp;quot;Inside noise&amp;quot; is two-sided, and is not the same as &amp;quot;useless&amp;quot;.&lt;/b&gt; The paired bootstrap uses a percentile 95% interval (rather than a studentized Ledoit-Wolf test) and tests &lt;code&gt;lo &amp;lt;= 0 &amp;lt;= hi&lt;/code&gt; symmetrically. On a short history or a high-volatility strategy, the interval can be wide enough that a genuinely sensible risk filter lands inside noise simply for lack of statistical power, or a pruned variant that happens to score slightly higher still sits inside noise. Treat a struck-through condition as a prompt to ask &amp;quot;what evidence do I actually have for this clause?&amp;quot;, not an automatic delete button.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Bisection assumes rough monotonicity.&lt;/b&gt; Rarely, higher slippage or a later start can alter cash availability enough to accidentally skip a large losing trade, causing total return to dip below idle cash at one level and bounce back slightly at a higher level. Bisection finds a small breaking level in &lt;code&gt;O(log B)&lt;/code&gt; runs, not an exhaustive scan of every integer.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Historical robustness cannot test unseen regimes.&lt;/b&gt; Surviving 50 bp of slippage and missing your top three stocks over the last decade says the historical edge was not concentrated in a handful of lucky fills. It cannot guarantee the underlying market effect will persist tomorrow.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Used alongside Rearview 1.1’s other checks, including &lt;b&gt;200 automatic execution copies&lt;/b&gt; after every run (&lt;a href=&quot;/docs/research/#copies&quot;&gt;Copies&lt;/a&gt;), cross-script trial accounting (&lt;a href=&quot;/docs/research/#the-evidence-budget&quot;&gt;The evidence budget&lt;/a&gt;), and a sealed lockbox, &lt;code&gt;Find What Matters&lt;/code&gt; turns two vague intuitions (&amp;quot;this script feels over-engineered&amp;quot; and &amp;quot;this edge feels fragile&amp;quot;) into concrete, reproducible numbers in under two seconds.&lt;/p&gt;&lt;p&gt;Read the full specification in &lt;a href=&quot;/docs/research/#what-matters&quot;&gt;What matters&lt;/a&gt;, or test how sensitive your trial count already is in the client-side &lt;a href=&quot;/is-my-backtest-luck/&quot;&gt;Deflated Sharpe Ratio calculator&lt;/a&gt;.&lt;/p&gt;</content></entry>
<entry><title>Point-in-time data vintages via reverse patches and Replay That Evening</title><id>https://rearview.page/writing/point-in-time-data-vintages/</id><link rel="alternate" type="text/html" href="https://rearview.page/writing/point-in-time-data-vintages/"/><published>2026-11-20T00:00:00Z</published><updated>2026-11-20T00:00:00Z</updated><summary>How Rearview keeps every version of your market data with zero-byte extensions and deterministic reverse byte deltas, and replays past live evenings to pinpoint why orders differ.</summary><content type="html">&lt;p&gt;Every systematic trader who updates market data daily runs into two quiet problems sooner or later.&lt;/p&gt;&lt;p&gt;First, &lt;b&gt;saved backtests stop reproducing&lt;/b&gt;. You run a strategy on Monday, save the result, update your database on Friday, and rerun it, only to watch three trades change because your data vendor corrected a historical close, delivered a dividend two days late, or back-adjusted a stock after a split.&lt;/p&gt;&lt;p&gt;Second, &lt;b&gt;live-order forensics breaks down&lt;/b&gt;. On Tuesday evening your system told you to buy 46 shares of a stock, so you placed the order Wednesday morning. A week later you check the backtest through Tuesday and it says you should have bought 52 shares, or never bought the stock at all. Was there a bug in your strategy, a mistake in your broker import, or did the data underneath you change after Tuesday night?&lt;/p&gt;&lt;p&gt;Keeping a full copy of a 5,000-security, 10-year binary store after every daily update multiplies disk usage rapidly. Throwing older versions away leaves you guessing whenever live trading and research diverge.&lt;/p&gt;&lt;p&gt;In Rearview 1.1, we solved both problems with &lt;b&gt;data versions&lt;/b&gt; and &lt;b&gt;Replay That Evening&lt;/b&gt; (rule C117): a four-tier vintage store built on a 180-line deterministic reverse byte-delta encoder, paired with a side-by-side live replay that traces differing orders back to the data changes behind them.&lt;/p&gt;&lt;h2 id=&quot;1-four-ways-to-keep-an-old-version-and-spend-zero-bytes-when-history-didnt-move&quot;&gt;1. Four ways to keep an old version (and spend zero bytes when history didn’t move)&lt;/h2&gt;&lt;p&gt;Most daily market updates do not change history at all: they simply append one new trading day (and maybe a new dividend or delisting on that day). Keeping a second copy of the entire 10-year store just to remember yesterday’s version would be wasteful.&lt;/p&gt;&lt;p&gt;Each Rearview store folder maintains a version index at &lt;code&gt;vintages/log.jsonl&lt;/code&gt;, oldest first, updated atomically via temp-file rename (&lt;code&gt;log.jsonl.tmp&lt;/code&gt; → &lt;code&gt;log.jsonl&lt;/code&gt;). Every line records the store’s content fingerprint, the timestamp it was recorded, its last trading day (&lt;code&gt;through&lt;/code&gt;), whether it was an &lt;code&gt;import&lt;/code&gt;, an append-only &lt;code&gt;days&lt;/code&gt; update, or a &lt;code&gt;history&lt;/code&gt; restatement (with the first date that changed), its payload SHA-256, and its byte length.&lt;/p&gt;&lt;p&gt;Each earlier version is stored in the smallest representation that rebuilds it cleanly:&lt;/p&gt;&lt;div class=&quot;table&quot;&gt;&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Storage mode (&lt;code&gt;kept&lt;/code&gt;)&lt;/th&gt;&lt;th&gt;When Rearview chooses it&lt;/th&gt;&lt;th&gt;Disk cost&lt;/th&gt;&lt;th&gt;How it rebuilds&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;current&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;The active store file itself&lt;/td&gt;&lt;td&gt;Existing store&lt;/td&gt;&lt;td&gt;Read directly&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;extended&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;The next update only added newer days and changed nothing up to the old last day&lt;/td&gt;&lt;td&gt;&lt;b&gt;0 bytes&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Truncate the newer base store at the old &lt;code&gt;through&lt;/code&gt; date (exact logical data and fingerprint)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;patch&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;History changed, and a reverse byte delta against the truncated new store is at most half the old file’s size&lt;/td&gt;&lt;td&gt;Compact delta (&lt;code&gt;&amp;lt;fp&amp;gt;.rvwpatch&lt;/code&gt;)&lt;/td&gt;&lt;td&gt;Truncate the base store at the old &lt;code&gt;through&lt;/code&gt; date, apply the reverse delta, verify SHA-256 (exact store file bytes)&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;store&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;History changed so broadly that a delta would exceed 50% of the old file&lt;/td&gt;&lt;td&gt;Full file (&lt;code&gt;&amp;lt;fp&amp;gt;.rvwstore&lt;/code&gt;, hard-linked when possible)&lt;/td&gt;&lt;td&gt;Read directly and verify payload SHA-256 (exact store file bytes)&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;p&gt;Two lifecycle rules keep the version graph consistent as updates accumulate:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Re-anchoring patches onto byte-exact bases.&lt;/b&gt; Notice the distinction in the table above: &lt;code&gt;patch&lt;/code&gt; and &lt;code&gt;store&lt;/code&gt; rebuild the exact &lt;code&gt;.rvwstore&lt;/code&gt; file bytes, while &lt;code&gt;extended&lt;/code&gt; rebuilds the exact logical data and fingerprint via truncation rather than guaranteeing identical file bytes. So if version &lt;code&gt;V1&lt;/code&gt; is stored as a reverse patch against &lt;code&gt;V2&lt;/code&gt;, and the next evening &lt;code&gt;V3&lt;/code&gt; arrives as an append-only update that turns &lt;code&gt;V2&lt;/code&gt; into &lt;code&gt;extended&lt;/code&gt;, Rearview reconstructs &lt;code&gt;V1&lt;/code&gt; from &lt;code&gt;V2&lt;/code&gt;, re-encodes &lt;code&gt;V1&lt;/code&gt; directly against &lt;code&gt;V3&lt;/code&gt;, and removes the old patch. Every patch stays anchored to a byte-exact base.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Reverting cleanly.&lt;/b&gt; If a later data update restores the exact fingerprint of an earlier version, Rearview removes that older version’s kept file and old log line, and records it at the end of &lt;code&gt;vintages/log.jsonl&lt;/code&gt; as &lt;code&gt;current&lt;/code&gt; with the new timestamp.&lt;/li&gt;&lt;/ul&gt;&lt;h2 id=&quot;2-why-reverse-patches-on-a-flat-binary-file&quot;&gt;2. Why reverse patches on a flat binary file?&lt;/h2&gt;&lt;p&gt;When designing data versioning for a local backtester, there are two obvious alternatives, and both carry real trade-offs:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Bitemporal databases (ArcticDB, SQLite, DuckDB, Parquet tables).&lt;/b&gt; Storing valid-time and transaction-time columns per row answers point-in-time queries natively, but adds indexing and query overhead to every ordinary backtest read. Rearview’s engine reads a flat, aligned columnar binary store (&lt;code&gt;market.rvwstore&lt;/code&gt;) straight into memory, running a 5,000-security × 10-year strategy cold from disk in &lt;b&gt;0.49 seconds&lt;/b&gt; (&lt;code&gt;cold_seconds = 0.487s&lt;/code&gt;). Keeping versioning as external reverse patches leaves everyday backtesting at &lt;b&gt;zero&lt;/b&gt; versioning overhead.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Off-the-shelf binary diff libraries (&lt;code&gt;bsdiff&lt;/code&gt;, &lt;code&gt;xdelta3&lt;/code&gt;, &lt;code&gt;zstd&lt;/code&gt; dictionaries).&lt;/b&gt; External binary-diff codecs either pull in heavy C dependencies, change output across library versions, or spend too much memory and CPU on 100 MB+ files. Because Rearview truncates the new store to the old version’s &lt;code&gt;through&lt;/code&gt; date before diffing, the two columnar binary files share the same calendar range and align almost everywhere, making a simple, deterministic 64-byte block matcher both fast and tiny.&lt;/li&gt;&lt;/ul&gt;&lt;h2 id=&quot;3-inside-the-reverse-byte-delta-format-rvwpatch&quot;&gt;3. Inside the reverse byte-delta format (&lt;code&gt;RVWPATCH&lt;/code&gt;)&lt;/h2&gt;&lt;p&gt;&lt;code&gt;crates/rearview-data/src/delta.rs&lt;/code&gt; implements a self-contained reverse byte-delta codec in ~180 lines of Rust (excluding tests).&lt;/p&gt;&lt;p&gt;Before diffing, Rearview truncates the &lt;b&gt;new&lt;/b&gt; store at the &lt;b&gt;old&lt;/b&gt; version’s last trading day (&lt;code&gt;through&lt;/code&gt;). The encoder indexes the truncated base in &lt;b&gt;64-byte blocks&lt;/b&gt; using a deterministic open-addressed hash table (fixed multiplier &lt;code&gt;0x0000_0100_0000_01b3&lt;/code&gt; and Fibonacci slot hashing, storing the first offset per block hash, with no random hasher seed). It then scans the old file with a rolling hash, and whenever a 64-byte block matches, extends the match both backward and forward byte by byte.&lt;/p&gt;&lt;p&gt;The resulting patch file (&lt;code&gt;vintages/&amp;lt;fingerprint&amp;gt;.rvwpatch&lt;/code&gt;) has an 88-byte little-endian header followed by a stream of just two opcodes with unsigned LEB128 varints:&lt;/p&gt;&lt;div class=&quot;code&quot;&gt;&lt;pre&gt;&lt;code&gt;offset  size  field
     0     8  magic &amp;quot;RVWPATCH&amp;quot;
     8     4  format version (1)
    12     4  reserved (0)
    16     8  old store file length
    24    32  SHA-256 of the old store file
    56    32  SHA-256 of the truncated base store file
    88     .  ops:  0, offset, length         (COPY from truncated base)
                    1, length, literal bytes  (ADD literal bytes)&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Three invariants keep patches safe to rely on months later:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Strict 50% size cutoff.&lt;/b&gt; The encoder aborts early and returns &lt;code&gt;None&lt;/code&gt; if &lt;code&gt;added_bytes&lt;/code&gt; exceeds half the old store’s length, and &lt;code&gt;keep&lt;/code&gt; checks again that the final encoded patch (&lt;code&gt;88&lt;/code&gt;-byte header + all &lt;code&gt;COPY&lt;/code&gt;/&lt;code&gt;ADD&lt;/code&gt; ops) is at most half the old store’s length. If hash collisions or widespread edits ever blow up a patch, Rearview silently falls back to keeping the whole &lt;code&gt;.rvwstore&lt;/code&gt; file.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Round-trip proof before saving.&lt;/b&gt; A patch is never written to disk on faith. Before keeping it, Rearview runs &lt;code&gt;apply_patch(&amp;amp;source, &amp;amp;patch)&lt;/code&gt; in memory and checks that the output equals the original old store file byte for byte.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Dual SHA-256 checks on rebuild.&lt;/b&gt; When rebuilding an old version (for &lt;code&gt;Check It Reproduces&lt;/code&gt;, &lt;code&gt;rvw rebuild&lt;/code&gt;, or &lt;code&gt;Replay That Evening&lt;/code&gt;), Rearview verifies both the SHA-256 of the truncated base input (&lt;code&gt;patch[56..88]&lt;/code&gt;) and the exact byte length plus SHA-256 of the reconstructed output (&lt;code&gt;patch[16..56]&lt;/code&gt;), with a recursion-depth guard (&lt;code&gt;versions.len() + 1&lt;/code&gt;) against cyclic base references.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;When you click &lt;b&gt;Check It Reproduces&lt;/b&gt; on a saved run whose data was later restated, Rearview matches the run’s recorded data fingerprint against &lt;code&gt;vintages/log.jsonl&lt;/code&gt;, rebuilds the exact vintage the run was made on, and checks that every output file still comes out byte for byte identical.&lt;/p&gt;&lt;h2 id=&quot;4-recording-time-vs-market-date-and-why-timezones-matter&quot;&gt;4. Recording time vs. market date (and why timezones matter)&lt;/h2&gt;&lt;p&gt;To answer &amp;quot;what did my data say on Tuesday evening?&amp;quot;, the store needs two different dates for every version:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;The market date (&lt;code&gt;through&lt;/code&gt;):&lt;/b&gt; the latest bar inside the database.&lt;/li&gt;&lt;li&gt;&lt;b&gt;The recording timestamp (&lt;code&gt;time&lt;/code&gt;):&lt;/b&gt; when that version was recorded, formatted as &lt;code&gt;YYYY-MM-DDTHH:MM:SS&lt;/code&gt; with &lt;code&gt;Z&lt;/code&gt; or a &lt;code&gt;±HH:MM&lt;/code&gt; offset.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Selecting the version known on &lt;code&gt;day&lt;/code&gt; (&lt;code&gt;known_on&lt;/code&gt;) filters versions whose &lt;b&gt;recorded date&lt;/b&gt; is on or before &lt;code&gt;day&lt;/code&gt;, then picks the one with the latest UTC instant. That timezone distinction matters in practice: an evening update run at 9:00 PM in Los Angeles on November 19 (&lt;code&gt;2026-11-19T21:00:00-08:00&lt;/code&gt;) is already 05:00 UTC on November 20, yet it belongs to the evening of November 19. The Mac app automatically records your local date and UTC offset. On the command line, &lt;code&gt;rvw import&lt;/code&gt; and &lt;code&gt;rvw update&lt;/code&gt; default to UTC (&lt;code&gt;Z&lt;/code&gt;), so pass &lt;code&gt;--at&lt;/code&gt; with your local offset for evening automation or when backfilling past exports you saved yourself.&lt;/p&gt;&lt;h2 id=&quot;5-replay-that-evening-and-the-two-bugs-real-yahoo-data-caught&quot;&gt;5. Replay That Evening, and the two bugs real Yahoo data caught&lt;/h2&gt;&lt;p&gt;With point-in-time vintages on disk, &lt;b&gt;Replay That Evening&lt;/b&gt; (&lt;code&gt;rvw live evening &amp;lt;dir&amp;gt; --store &amp;lt;store&amp;gt; --day YYYY-MM-DD --out &amp;lt;folder&amp;gt;&lt;/code&gt;) takes any past trading day of a live account and runs two replays side by side:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Then:&lt;/b&gt; rebuilds the version recorded last by the end of that evening, cuts it at &lt;code&gt;day&lt;/code&gt; (if it extended past &lt;code&gt;day&lt;/code&gt;), replays your actual broker fills through &lt;code&gt;day&lt;/code&gt;, and computes the next session’s orders.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Now:&lt;/b&gt; cuts today’s store at &lt;code&gt;day&lt;/code&gt;, replays those exact same broker fills through &lt;code&gt;day&lt;/code&gt;, and computes the next session’s orders.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;When the two order lists match, whatever your vendor changed since that evening made no difference to your live trading. When an order differs in presence or share count (keyed by security, action, order type, limit and stop price), Rearview explains &lt;b&gt;why&lt;/b&gt;.&lt;/p&gt;&lt;p&gt;Our first implementation of that explanation looked clean in unit tests, and failed two real-world checks the moment we dogfooded it on actual Yahoo Finance exports:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;A 2004 re-adjustment hid a split three days earlier.&lt;/b&gt; The first version printed the earliest historical change on the differing order’s security. Because Yahoo Finance’s adjusted files rescale prices all the way back to a ticker’s start when a split arrives, the UI blamed a price change in 2004 instead of the GOOGL split that happened three days before that evening.&lt;/li&gt;&lt;li&gt;&lt;b&gt;An untouched stock’s buy size changed because a different held stock split.&lt;/b&gt; When a stock the account &lt;b&gt;already held&lt;/b&gt; that evening had a late split or corrected close, the account’s closing equity shifted. Because the strategy sized new buys as a percentage of equity, the share count of an unrelated buy order moved, and looking only at changes on the buy order’s own security reported no changes at all.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;Fixing both taught us to collect &lt;b&gt;every&lt;/b&gt; data change that can reach an order across four explicit causal scopes, sorted &lt;b&gt;most recent first&lt;/b&gt; by the day a run could first observe it:&lt;/p&gt;&lt;div class=&quot;table&quot;&gt;&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Causal scope (&lt;code&gt;via&lt;/code&gt;)&lt;/th&gt;&lt;th&gt;What it covers&lt;/th&gt;&lt;th&gt;Tie-break priority&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;own&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Price, split, dividend, delisting, membership, ticker or series changes on the differing order’s own security&lt;/td&gt;&lt;td&gt;1&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;held&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Changes on any security the account held at the close of &lt;code&gt;day&lt;/code&gt; on either the &amp;quot;then&amp;quot; or &amp;quot;now&amp;quot; replay (marked &lt;b&gt;Held that evening&lt;/b&gt; in the app)&lt;/td&gt;&lt;td&gt;2&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;market&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;Store-wide changes not tied to a single security (&lt;code&gt;CALENDAR_CHANGED&lt;/code&gt;, &lt;code&gt;RATES_CHANGED&lt;/code&gt;, named series added/removed, or unitemised summary changes)&lt;/td&gt;&lt;td&gt;3&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;&lt;code&gt;late&lt;/code&gt;&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&lt;code&gt;DAYS_LATE&lt;/code&gt;: trading days today’s data has through &lt;code&gt;day&lt;/code&gt; that were missing on that evening because the data hadn’t been updated yet&lt;/td&gt;&lt;td&gt;4&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;p&gt;Re-running the same real-data dogfood test immediately surfaced the GOOGL split on both the GOOGL order (&lt;code&gt;own&lt;/code&gt;) and the resized buy order (&lt;code&gt;held&lt;/code&gt;, marked &lt;b&gt;Held that evening&lt;/b&gt;), while keeping older 2004 re-adjustments neatly tucked behind it in the full change list.&lt;/p&gt;&lt;h2 id=&quot;6-speed-and-randomized-verification&quot;&gt;6. Speed and randomized verification&lt;/h2&gt;&lt;p&gt;Point-in-time versioning is only practical if rebuilding an older store feels instantaneous during an evening check. On Rearview 1.1.0, our single-threaded &lt;code&gt;rebuild_vintage_seconds&lt;/code&gt; benchmark cuts a &lt;b&gt;5,000-security × 10-year (2,520-day)&lt;/b&gt; store to four-fifths of its history (&lt;code&gt;2,016&lt;/code&gt; days), applies a reverse &lt;code&gt;.rvwpatch&lt;/code&gt;, verifies both SHA-256 digests, and reads the rebuilt store into memory in &lt;b&gt;0.79 seconds&lt;/b&gt; (&lt;code&gt;0.793s&lt;/code&gt; measured on an Apple M5 Max; speed budget &lt;code&gt;2.5s&lt;/code&gt;, see &lt;a href=&quot;/docs/verification/#speed&quot;&gt;How Rearview is checked&lt;/a&gt;).&lt;/p&gt;&lt;p&gt;Before every release, &lt;code&gt;xcheck/vintages.py&lt;/code&gt; stress-tests the vintage index and reverse-patch engine across &lt;b&gt;40 randomized update sequences&lt;/b&gt; spanning &lt;b&gt;303 successful import/update steps&lt;/b&gt; (plus refused partial updates) and &lt;b&gt;242 kept historical versions&lt;/b&gt; (282 total versions including the 40 current stores), mixing corrected bars, dropped bars, rescaled prices, late splits, doubled prices, partial daily updates and refused updates. After every step, the harness verifies the index’s ordering and storage modes, rebuilds every historical version (checking exact file bytes for every patch and whole store, and exact fingerprint for every extended version), and checks &lt;code&gt;known_on&lt;/code&gt; selection for every date.&lt;/p&gt;&lt;h2 id=&quot;7-honest-limits&quot;&gt;7. Honest limits&lt;/h2&gt;&lt;p&gt;Stating what a mechanism cannot do is as important as documenting what it does:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;No retroactive time travel without saved exports.&lt;/b&gt; A store only knows versions recorded from the time versioning began (or older exports you backfill yourself with &lt;code&gt;--at&lt;/code&gt;). If a vendor silently corrected a close last year before you imported it, no local versioning system can recover what the vendor never gave you.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Consecutive history restatements chain patches.&lt;/b&gt; While append-only updates re-anchor older patches directly onto the new store, a sequence of back-to-back history restatements (common if you update back-adjusted per-symbol files every week as new dividends arrive) builds a chain of reverse patches until an edit crosses the 50% cutoff and writes a whole &lt;code&gt;.rvwstore&lt;/code&gt; file. Using as-traded bars with a separate &lt;code&gt;actions.csv&lt;/code&gt; keeps almost every daily update append-only (&lt;code&gt;extended&lt;/code&gt;, 0 bytes).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Causal candidates, not counterfactual proof.&lt;/b&gt; &lt;code&gt;Replay That Evening&lt;/code&gt; lists every change on the order’s security (&lt;code&gt;own&lt;/code&gt;), held securities (&lt;code&gt;held&lt;/code&gt;), store-wide changes (&lt;code&gt;market&lt;/code&gt;) and late days (&lt;code&gt;late&lt;/code&gt;). It does not run an ablation over every individual bar edit to prove which of two simultaneous edits flipped a signal. Moreover, changes to a third security that was neither ordered nor held (such as an unheld market-regime ETF like &lt;code&gt;SPY&lt;/code&gt; whose restated close flipped a regime filter, or an unheld stock whose rank leapfrogged your candidate) appear in the top-level &lt;code&gt;history&lt;/code&gt; array of &lt;code&gt;evening.json&lt;/code&gt; rather than directly inside that order’s &lt;code&gt;changes&lt;/code&gt; list.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Market data is only half of live reality.&lt;/b&gt; Versioning proves what your &lt;b&gt;data&lt;/b&gt; said that evening; it cannot detect a broker-specific corporate-action booking or an intraday fill discrepancy, which is what the live account’s Fills and Statement tabs track separately.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Read the full documentation in &lt;a href=&quot;/docs/import/#keeping-your-data-up-to-date&quot;&gt;Keeping your data up to date&lt;/a&gt; and &lt;a href=&quot;/docs/live/#replay-that-evening&quot;&gt;Replay That Evening&lt;/a&gt;, or see every release check at &lt;a href=&quot;/docs/verification/&quot;&gt;How Rearview is checked&lt;/a&gt;.&lt;/p&gt;</content></entry>
<entry><title>Checking a portfolio backtester against a second engine, vectorbt, bt and LEAN</title><id>https://rearview.page/writing/checking-a-backtester/</id><link rel="alternate" type="text/html" href="https://rearview.page/writing/checking-a-backtester/"/><published>2026-11-20T00:00:00Z</published><updated>2026-11-20T00:00:00Z</updated><summary>How we verify a Rust portfolio backtester with hand-worked cases, differential testing against a Python reference engine, external anchors in vectorbt, bt and LEAN, and cut-date look-ahead replays.</summary><content type="html">&lt;p&gt;November 2026 · Figures below are from the Rearview 1.0.2 release check (18 November 2026); the latest numbers live at &lt;a href=&quot;/docs/verification/&quot;&gt;How Rearview is checked&lt;/a&gt;.&lt;/p&gt;&lt;p&gt;Small unit tests on short arrays catch obvious coding mistakes, such as an off-by-one loop or a broken moving average, but they rarely catch what actually breaks a portfolio backtest: an overnight gap that exhausts cash mid-session, a reverse split colliding with a short position, a delisted stock that quietly disappears from a universe, or a subtle look-ahead leak in a cross-sectional rank.&lt;/p&gt;&lt;p&gt;When building Rearview, we wanted a stricter release bar: before every release, the engine has to prove it follows its own written rules, matches a separate implementation written without reading its source code, agrees with three open-source backtesters on the accounting and mechanics they share, and cannot see future prices across thousands of cut-date replays.&lt;/p&gt;&lt;p&gt;This note describes the four verification layers in our release CI, the recorded 1.0.2 figures, and the design differences we documented when anchoring Rearview against &lt;code&gt;vectorbt&lt;/code&gt;, &lt;code&gt;bt&lt;/code&gt; and QuantConnect’s &lt;code&gt;LEAN&lt;/code&gt;.&lt;/p&gt;&lt;h2 id=&quot;1-written-rules-and-104-hand-worked-cases&quot;&gt;1. Written rules and 104 hand-worked cases&lt;/h2&gt;&lt;p&gt;Verification starts with a specification, not code. Rearview’s engine follows numbered written rules covering order fills, cash and share accounting, splits, dividends, delistings, Reg T margin, short borrowing, multi-strategy sleeves and shared cash pools, 365-day coin calendars, daily-settled futures chains with rolls, multi-exchange holidays, multi-currency conversion and intraday bars.&lt;/p&gt;&lt;p&gt;From those rules alone, we worked out 104 small scenarios by hand (80 single-strategy and 24 multi-strategy portfolio cases): every fill price, share count, cash balance, dividend, borrow fee and closing equity written down in exact integer millionths of a dollar (&lt;code&gt;$0.000001&lt;/code&gt;). The Rust engine must reproduce every expected CSV file byte for byte.&lt;/p&gt;&lt;p&gt;Hand-worked cases catch an engine that disagrees with its specification, but they have two blind spots: nobody writes hand-worked cases for every strange interaction of features, and hand-worked cases cannot catch a bug in the rules themselves.&lt;/p&gt;&lt;h2 id=&quot;2-a-second-engine-from-the-spec-alone-2200-random-scenarios-0-differences&quot;&gt;2. A second engine from the spec alone (2,200 random scenarios, 0 differences)&lt;/h2&gt;&lt;p&gt;To catch bugs in the rare combinations nobody writes by hand, we built a second engine (&lt;code&gt;refbt&lt;/code&gt;) in Python from the written rules and hand-worked cases alone, without reading the Rust engine’s source code. Both engines and the harness live in our internal release CI.&lt;/p&gt;&lt;p&gt;A second implementation written by the same team does not prove the written rules are right on their own, since both engines follow the same spec; that is what Layer 3 is for. What it catches reliably are translation mistakes, edge-case divergences and multi-feature interactions.&lt;/p&gt;&lt;p&gt;Before every release, the cross-check harness generates 2,200 random scenarios across five profiles:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Ties&lt;/b&gt; (every 10th scenario): many candidates with identical rank scores, exercising deterministic tie-breaking.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Stress&lt;/b&gt; (high leverage): margin calls, forced liquidations and account wipe-outs.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Portfolios&lt;/b&gt;: two or three strategies with independent signals, random weights, fixed sleeves (half with rebalancing that sells holdings to restore weights) or one shared pool of cash.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Lean&lt;/b&gt;: most optional features switched off, isolating core accounting.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Normal&lt;/b&gt;: random combinations of long and short entries, adds, scale-outs, stops, targets, time exits, corporate actions, late listings, delistings, settings changing by date and multiple markets.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;The Rust CLI (&lt;code&gt;rvw&lt;/code&gt;) runs on the imported binary store while &lt;code&gt;refbt&lt;/code&gt; reads the raw CSV folder directly, so the check exercises the importer too. Every canonical output file (&lt;code&gt;orders.csv&lt;/code&gt;, &lt;code&gt;fills.csv&lt;/code&gt;, &lt;code&gt;events.csv&lt;/code&gt;, &lt;code&gt;trades.csv&lt;/code&gt;, &lt;code&gt;daily.csv&lt;/code&gt;) is compared byte for byte. On Rearview 1.0.2, all 2,200 scenarios matched with &lt;b&gt;0 differences&lt;/b&gt;, and both engines agreed on all &lt;b&gt;368 invalid and boundary configurations&lt;/b&gt; (accepting and rejecting the exact same set).&lt;/p&gt;&lt;p&gt;The same differential check covers &lt;b&gt;760 live-account replays&lt;/b&gt; (45,301 broker fills, including partial fills, late sells, position reversals, deposits, withdrawals, coin fills and fractional shares), random-entry baselines, stress tests and performance attribution.&lt;/p&gt;&lt;p&gt;Byte-for-byte agreement is possible because cash and prices are stored in exact integer millionths of a dollar (&lt;code&gt;$0.000001&lt;/code&gt;), never binary floating point, and parallel work across securities reduces in a fixed order. The same run produces the exact same bytes on one core or eighteen.&lt;/p&gt;&lt;h2 id=&quot;3-anchoring-against-vectorbt-bt-and-quantconnect-lean-220-of-220&quot;&gt;3. Anchoring against vectorbt, bt and QuantConnect LEAN (220 of 220)&lt;/h2&gt;&lt;p&gt;Two engines written from the same specification can still share a mistake in the specification itself. To test our accounting and portfolio rules against outside implementations, we run 220 randomised scenarios through three independent open-source backtesters:&lt;/p&gt;&lt;div class=&quot;table&quot;&gt;&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Outside engine&lt;/th&gt;&lt;th&gt;What it anchors&lt;/th&gt;&lt;th&gt;How it is driven&lt;/th&gt;&lt;th&gt;Scenarios&lt;/th&gt;&lt;th&gt;Fills checked&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;vectorbt&lt;/b&gt; (1.1.1)&lt;/td&gt;&lt;td&gt;Long entries, exits, costs, fractional shares, 365-day coin years, AUD account holding USD stocks, 5-minute intraday bars&lt;/td&gt;&lt;td&gt;Pre-shifted share orders (and pre-converted FX prices for AUD) to verify fill accounting, equity and Sharpe/Sortino&lt;/td&gt;&lt;td&gt;80 of 80&lt;/td&gt;&lt;td&gt;6,644&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;bt&lt;/b&gt; (1.3.0)&lt;/td&gt;&lt;td&gt;Weekly, monthly and quarterly rebalancing to equal or fixed weights, whole and fractional shares&lt;/td&gt;&lt;td&gt;50 orders-mode fill replays plus 50 weights-mode runs where &lt;code&gt;bt&lt;/code&gt; picks rebalance dates, ranks and shares independently&lt;/td&gt;&lt;td&gt;100 of 100&lt;/td&gt;&lt;td&gt;19,731&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;&lt;b&gt;QuantConnect LEAN&lt;/b&gt; (18139)&lt;/td&gt;&lt;td&gt;Short positions, 1.5x margin, forward/reverse splits, dividends and acquisition delistings&lt;/td&gt;&lt;td&gt;Replays Rearview’s entry and exit fills while &lt;code&gt;LEAN&lt;/code&gt; independently applies splits, dividends, delistings, margin and cash&lt;/td&gt;&lt;td&gt;40 of 40&lt;/td&gt;&lt;td&gt;796&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;p&gt;In &lt;code&gt;bt&lt;/code&gt;, whole-share runs without costs show a maximum relative closing-equity difference of &lt;code&gt;1.97e-15&lt;/code&gt; on orders and &lt;code&gt;9.89e-16&lt;/code&gt; on weights (fractional-share weights stay within &lt;code&gt;8.06e-08&lt;/code&gt;). In &lt;code&gt;vectorbt&lt;/code&gt; without costs, trades match exactly and Sharpe and Sortino ratios agree within &lt;code&gt;1e-6&lt;/code&gt; relative (using 365 days for coin runs and 252 for stock runs). In &lt;code&gt;LEAN&lt;/code&gt;, shares held, cash and equity agree after every close across 43 splits, 115 dividends and 18 delistings (within &lt;code&gt;$0.000001&lt;/code&gt; after a reverse split, where &lt;code&gt;LEAN&lt;/code&gt; computes fractional cash-in-lieu through a binary float, and with equity alone compared on a delisting’s final close since &lt;code&gt;LEAN&lt;/code&gt; books the close at the next open).&lt;/p&gt;&lt;p&gt;Stops, trailing stops, limit orders, multi-strategy portfolios, margin calls and performance-delisting haircuts are verified by &lt;code&gt;refbt&lt;/code&gt; and hand-worked cases rather than these outside anchors.&lt;/p&gt;&lt;h3 id=&quot;where-the-engines-disagree-by-design&quot;&gt;Where the engines disagree by design&lt;/h3&gt;&lt;p&gt;The most useful outcome of building external anchors was documenting the exact places where mature backtesters make different legitimate choices. Each of these has to be either aligned in the scenario config or left outside the anchor and documented explicitly:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Rebalancing on the last bar (&lt;code&gt;bt&lt;/code&gt;):&lt;/b&gt; &lt;code&gt;bt&lt;/code&gt;’s &lt;code&gt;RunPeriod&lt;/code&gt; algos do not rebalance on the dataset’s final date unless &lt;code&gt;run_on_last_date&lt;/code&gt; is set, whereas Rearview can emit rebalance orders on the last day.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Decision bar vs. next open (&lt;code&gt;bt&lt;/code&gt;):&lt;/b&gt; &lt;code&gt;bt&lt;/code&gt; fills at the same bar’s close on which it computes target weights. Rearview decides at a close and fills at the next session’s open by default, so the weights anchor explicitly selects &lt;code&gt;same_close_approx&lt;/code&gt; while the orders anchor covers next-open fills.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Commission-aware sizing (&lt;code&gt;bt&lt;/code&gt;):&lt;/b&gt; &lt;code&gt;bt&lt;/code&gt; shrinks a buy (or enlarges a sell) so that the commission fits inside the target dollar allocation. Rearview sizes from the target weight alone and only shrinks a buy if cash runs short.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Negative cash vs. shrunk orders (&lt;code&gt;bt&lt;/code&gt; and &lt;code&gt;LEAN&lt;/code&gt;):&lt;/b&gt; When target buys slightly exceed available cash, &lt;code&gt;bt&lt;/code&gt; lets cash go negative, &lt;code&gt;LEAN&lt;/code&gt; rejects an order that exceeds buying power, and Rearview shrinks the order’s share count to fit available cash.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Short borrow fee timing (&lt;code&gt;LEAN&lt;/code&gt;):&lt;/b&gt; &lt;code&gt;LEAN&lt;/code&gt; starts charging a short’s borrow fee from the second day it observes the position, based on that day’s value; Rearview charges from the first night held, based on the previous close.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Delisting close timing and haircuts (&lt;code&gt;LEAN&lt;/code&gt;):&lt;/b&gt; &lt;code&gt;LEAN&lt;/code&gt; closes every delisted position at its last traded price at the start of the next day. Rearview closes at the delisting day’s close and applies a configurable haircut (default 30% on NYSE, 55% on Nasdaq) when the delisting was for poor performance rather than an acquisition.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Margin liquidation order (&lt;code&gt;LEAN&lt;/code&gt;):&lt;/b&gt; On a margin breach, &lt;code&gt;LEAN&lt;/code&gt; trims positions partially back to its maintenance threshold; Rearview liquidates whole positions, largest unrealised loss first, against FINRA-style maintenance rules.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;None of these are bugs in &lt;code&gt;bt&lt;/code&gt; or &lt;code&gt;LEAN&lt;/code&gt;. They are design choices, and moving a strategy from one engine to another without accounting for them causes equity curves to diverge even on identical data.&lt;/p&gt;&lt;h2 id=&quot;4-look-ahead-replay-planted-leaks-and-fuzzing&quot;&gt;4. Look-ahead replay, planted leaks and fuzzing&lt;/h2&gt;&lt;p&gt;A backtest that peeks at tomorrow’s prices looks wonderful and is worthless. Before every release, Rearview tests the entire pipeline (split and dividend adjustment, point-in-time index membership, weekly and monthly bar aggregation, cross-sectional ranking, strategy evaluation and simulation) in three ways:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Cut-date replay (10,924 cut-date runs across 71 scripts and 4 portfolios):&lt;/b&gt; History is truncated at cut dates landing on splits, dividends, delistings, index changes, month ends, late listings and inside intraday sessions (first bars, last bars and half days). Every decision and fill up to each cut must come out identical to the run on the full history (3,393,969 fills and 1,525,185 trades replayed on 1.0.2 with zero differences). Scans are replayed the same way across 23 cut dates and 9,880 rows.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Future perturbation:&lt;/b&gt; Randomly altering every price after a cut date must not change a single decision before it.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Planted leaks:&lt;/b&gt; To prove the detector actually works, the suite feeds intentionally leaky inputs (signals reading tomorrow’s close, adjustments computed from future splits, universes that know future delistings) and requires the test to catch every single one.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;Finally, the CSV importer, converters and store reader are fuzzed with &lt;b&gt;200,000 broken inputs&lt;/b&gt; (flipped bytes, truncated files, shuffled rows, NaN, negative and huge numbers, renamed columns and damaged stores) with 0 crashes and 0 unhandled failures, and &lt;b&gt;104 golden replays&lt;/b&gt; check that saved runs still reproduce byte for byte across releases.&lt;/p&gt;&lt;h2 id=&quot;what-this-still-cannot-catch&quot;&gt;What this still cannot catch&lt;/h2&gt;&lt;p&gt;Verification only matters if you state its boundaries plainly:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;A rule that both Rearview engines and every outside engine get wrong in the same way.&lt;/b&gt; Only hand-worked cases and domain review guard against that.&lt;/li&gt;&lt;li&gt;&lt;b&gt;A leak inside your own data.&lt;/b&gt; If you import today’s S&amp;amp;P 500 constituents or pre-adjusted vendor prices without corporate actions, the engine cannot invent missing delistings. Rearview’s import report flags what the files cannot tell it.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Markets not yet anchored to an outside engine.&lt;/b&gt; Futures rolls, forex carry and multi-exchange calendars are verified by hand-worked cases and the Python reference engine, not yet against a third-party engine.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Real broker nuances.&lt;/b&gt; The short-sale price test (Reg SHO Rule 201) is not modelled, and a broker can still book a corporate action or borrow rate differently from your market data.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;The latest verification figures and every speed budget are published automatically from our release check at &lt;a href=&quot;/docs/verification/&quot;&gt;How Rearview is checked&lt;/a&gt;. If you ever find a result that looks wrong, Rearview’s &lt;b&gt;Help ▸ Report a Wrong Result…&lt;/b&gt; bundles the script, settings and data fingerprint (without your bars) so we can replay it and add a new reference case.&lt;/p&gt;</content></entry>
<entry><title>Why AI coding agents p-hack backtests, and how a chained tries log catches it</title><id>https://rearview.page/writing/ai-agents-and-backtest-overfitting/</id><link rel="alternate" type="text/html" href="https://rearview.page/writing/ai-agents-and-backtest-overfitting/"/><published>2026-11-20T00:00:00Z</published><updated>2026-11-20T00:00:00Z</updated><summary>What 300 automated iterations do to the Deflated Sharpe Ratio, why you need PBO too, and how a token-hashed, SHA-256-chained tries log tracks every try.</summary><content type="html">&lt;p&gt;Give a coding agent a folder of historical prices, a strategy script and a common prompt: &amp;quot;Test a short-term pullback strategy on S&amp;amp;P 500 stocks and improve its risk-adjusted return.&amp;quot;&lt;/p&gt;&lt;p&gt;Twenty minutes later the agent returns a clean script with a smooth equity curve, a tidy git diff and a strong Sharpe ratio. You can read every line, confirm there is no look-ahead bias, verify the commissions, and still be looking at noise.&lt;/p&gt;&lt;p&gt;What disappeared along the way are the 180 parameter combinations, alternative indicators, regime filters and exit rules the agent tried in its loop before keeping the one that survived. An agent does not need to introduce a bug to mislead you: automating iteration compresses a month of human data-mining into an afternoon of &lt;b&gt;selection bias under multiple testing&lt;/b&gt;.&lt;/p&gt;&lt;h2 id=&quot;the-maths-of-an-afternoon-of-iteration&quot;&gt;The maths of an afternoon of iteration&lt;/h2&gt;&lt;p&gt;Under i.i.d. normal returns with no autocorrelation or regime shift (Lo, 2002), the standard error of an annualised Sharpe ratio over ten years is roughly &lt;code&gt;1 / sqrt(10) ≈ 0.32&lt;/code&gt;. On a single honest hypothesis, an observed Sharpe ratio of &lt;code&gt;0.95&lt;/code&gt; sits roughly three standard errors above zero (&lt;code&gt;Phi(z) &amp;gt; 99%&lt;/code&gt;).&lt;/p&gt;&lt;p&gt;Now let an agent run &lt;b&gt;300 variations&lt;/b&gt; on that same ten-year history. Even if every variation has zero true edge, picking the maximum of 300 noisy draws pushes the best observed Sharpe upward rapidly.&lt;/p&gt;&lt;p&gt;David Bailey and Marcos López de Prado formalised this in the &lt;b&gt;Deflated Sharpe Ratio&lt;/b&gt; (2014). Given &lt;code&gt;N&lt;/code&gt; trials with average return correlation &lt;code&gt;ρ&lt;/code&gt; and cross-trial Sharpe standard deviation &lt;code&gt;σ&lt;/code&gt;, the effective number of independent trials is:&lt;/p&gt;&lt;div class=&quot;code&quot;&gt;&lt;pre&gt;&lt;code&gt;N_eff = ρ + (1 - ρ) * N
SR0   = σ * ((1 - γ) * Phi^-1(1 - 1 / N_eff) + γ * Phi^-1(1 - 1 / (N_eff * e)))
z     = (SR - SR0) * sqrt(T - 1) / sqrt(1 - g3 * SR + (g4 - 1) / 4 * SR^2)
DSR   = Phi(z)&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;where &lt;code&gt;γ ≈ 0.5772&lt;/code&gt; is the Euler-Mascheroni constant, &lt;code&gt;T&lt;/code&gt; is the number of return observations, &lt;code&gt;g3&lt;/code&gt; is daily skewness and &lt;code&gt;g4&lt;/code&gt; is daily non-excess kurtosis (&lt;code&gt;3&lt;/code&gt; for a normal distribution). Note the scale: &lt;code&gt;SR&lt;/code&gt;, &lt;code&gt;SR0&lt;/code&gt; and &lt;code&gt;σ&lt;/code&gt; enter the &lt;code&gt;z&lt;/code&gt; denominator in &lt;b&gt;per-period (daily) units&lt;/b&gt; (&lt;code&gt;annual ÷ sqrt(252)&lt;/code&gt; for daily stock returns), not annualised units.&lt;/p&gt;&lt;p&gt;Using one fixed return profile across all three rows (&lt;code&gt;T = 2,520&lt;/code&gt; daily returns, &lt;code&gt;g3 = -0.4&lt;/code&gt;, &lt;code&gt;g4 = 6.0&lt;/code&gt;, with &lt;code&gt;SR&lt;/code&gt;, &lt;code&gt;SR0&lt;/code&gt; and &lt;code&gt;σ&lt;/code&gt; shown annualised):&lt;/p&gt;&lt;div class=&quot;table&quot;&gt;&lt;table&gt;&lt;thead&gt;&lt;tr&gt;&lt;th&gt;Variations tried (N)&lt;/th&gt;&lt;th&gt;Effective trials (N_eff)&lt;/th&gt;&lt;th&gt;Luck’s best Sharpe (SR0)&lt;/th&gt;&lt;th&gt;Observed Sharpe&lt;/th&gt;&lt;th&gt;Deflated Sharpe (DSR)&lt;/th&gt;&lt;th&gt;Verdict&lt;/th&gt;&lt;/tr&gt;&lt;/thead&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td&gt;1 (first try)&lt;/td&gt;&lt;td&gt;1.0&lt;/td&gt;&lt;td&gt;0.00&lt;/td&gt;&lt;td&gt;0.95&lt;/td&gt;&lt;td&gt;99.8%&lt;/td&gt;&lt;td&gt;Probably not luck&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;50 tweaks (&lt;code&gt;σ=0.35, ρ=0.30&lt;/code&gt;)&lt;/td&gt;&lt;td&gt;35.3&lt;/td&gt;&lt;td&gt;0.75&lt;/td&gt;&lt;td&gt;1.20&lt;/td&gt;&lt;td&gt;91.9%&lt;/td&gt;&lt;td&gt;Can’t tell&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;300 agent tries (&lt;code&gt;σ=0.50, ρ=0.15&lt;/code&gt;)&lt;/td&gt;&lt;td&gt;255.2&lt;/td&gt;&lt;td&gt;1.42&lt;/td&gt;&lt;td&gt;1.30&lt;/td&gt;&lt;td&gt;35.3%&lt;/td&gt;&lt;td&gt;&lt;b&gt;Probably luck&lt;/b&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;/div&gt;&lt;p&gt;One important subtlety: &lt;code&gt;SR0&lt;/code&gt; is sensitive to the cross-trial spread &lt;code&gt;σ&lt;/code&gt;. Under pure zero-skill noise over ten years (&lt;code&gt;SE ≈ 0.32&lt;/code&gt;), 300 random tries have a smaller spread around &lt;code&gt;σ = 0.32&lt;/code&gt;, which puts &lt;code&gt;SR0&lt;/code&gt; near &lt;code&gt;0.91&lt;/code&gt; and leaves a &lt;code&gt;1.30&lt;/code&gt; Sharpe at &lt;code&gt;88.6%&lt;/code&gt; (&lt;code&gt;Can’t tell&lt;/code&gt;). In real agent exploration, where the loop mixes working ideas, inverted rules and high-turnover variants that bleed commissions, cross-trial spread widens quickly (&lt;code&gt;σ = 0.50&lt;/code&gt; or more), lifting &lt;code&gt;SR0&lt;/code&gt; to &lt;code&gt;1.42&lt;/code&gt; and dropping that same &lt;code&gt;1.30&lt;/code&gt; Sharpe to &lt;code&gt;35.3%&lt;/code&gt; (&lt;b&gt;Probably luck&lt;/b&gt;). Remember too that &lt;code&gt;Phi(z)&lt;/code&gt; is a frequentist test score under i.i.d. assumptions (real serial correlation and regime shifts widen uncertainty further), not an unconditional probability that a strategy will make money live.&lt;/p&gt;&lt;p&gt;You can test how sensitive DSR is to &lt;code&gt;N&lt;/code&gt;, &lt;code&gt;σ&lt;/code&gt; and &lt;code&gt;ρ&lt;/code&gt; in our browser-based &lt;a href=&quot;/is-my-backtest-luck/&quot;&gt;Deflated Sharpe Ratio calculator&lt;/a&gt;.&lt;/p&gt;&lt;h2 id=&quot;why-deflated-sharpe-alone-is-not-enough&quot;&gt;Why Deflated Sharpe alone is not enough&lt;/h2&gt;&lt;p&gt;Even when a rule family clears Deflated Sharpe, the specific parameter setting you choose can still be noise.&lt;/p&gt;&lt;p&gt;In Rearview’s synthetic sample project (10 years of generated prices, not a real market), &lt;code&gt;down_days_overtuned&lt;/code&gt; searches 50 combinations of two parameters. Its Deflated Sharpe is &lt;b&gt;97%&lt;/b&gt; (the underlying rule family earns something on the synthetic series even after counting all 50 trials), yet its &lt;b&gt;Probability of Backtest Overfitting (PBO)&lt;/b&gt; is &lt;b&gt;82%&lt;/b&gt;: across all half-and-half combinations of time blocks (CSCV, Bailey et al., 2017), the setting that looked best in one half ranked below the median in the other half 82% of the time.&lt;/p&gt;&lt;p&gt;Because you trade the winning setting rather than the average of all 50, Rearview pairs both checks and gives one verdict: &lt;b&gt;Probably luck&lt;/b&gt;.&lt;/p&gt;&lt;h2 id=&quot;why-manual-trial-counting-fails&quot;&gt;Why manual trial counting fails&lt;/h2&gt;&lt;p&gt;The Deflated Sharpe formula takes twenty lines of code. The hard part in practice is knowing &lt;code&gt;N&lt;/code&gt;.&lt;/p&gt;&lt;p&gt;Humans forget last Tuesday’s runs, delete weak equity curves, and undercount how many ideas they glanced at. AI coding agents amplify that gap: in a typical agent loop, intermediate runs live in &lt;code&gt;/tmp&lt;/code&gt;, terminal scrollback, or overwritten notebooks, so by the time you review the winning script, the trial count is gone.&lt;/p&gt;&lt;h2 id=&quot;a-pattern-for-automatic-trial-accounting-and-how-rearview-implements-it&quot;&gt;A pattern for automatic trial accounting (and how Rearview implements it)&lt;/h2&gt;&lt;p&gt;Any systematic research setup that uses coding agents needs three properties: the agent runs the same engine as the human, every try is appended to a persistent log outside the run folders, and the most recent years are held back in a lockbox until the loop ends.&lt;/p&gt;&lt;p&gt;Here is how we built that into Rearview. Strategies are plain-text &lt;code&gt;.rvw&lt;/code&gt; files in a regular folder, the &lt;a href=&quot;/docs/language/&quot;&gt;language reference&lt;/a&gt; fits on one page (&lt;code&gt;File ▸ Save Language Reference to Project&lt;/code&gt;), and &lt;code&gt;rvw check &amp;lt;script.rvw&amp;gt; --json&lt;/code&gt; gives an agent structured line-and-column diagnostics so it can fix syntax errors on its own. When the agent tests a script inside the project with:&lt;/p&gt;&lt;div class=&quot;code&quot;&gt;&lt;pre&gt;&lt;code&gt;rvw run --project . --script pullback.rvw&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;(or &lt;code&gt;rvw search&lt;/code&gt; / &lt;code&gt;rvw walkforward&lt;/code&gt; with &lt;code&gt;--project . --script ... --space ...&lt;/code&gt;), the engine appends the run to the project’s &lt;code&gt;tries.jsonl&lt;/code&gt; under an exclusive file lock, right beside every run and preview made in the Mac app.&lt;/p&gt;&lt;p&gt;Four engineering details keep the log useful during messy iteration:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Token-hashed try keys.&lt;/b&gt; Each try is keyed by the first 32 hex digits of the SHA-256 of &lt;code&gt;{&amp;quot;code&amp;quot;, &amp;quot;params&amp;quot;, &amp;quot;run&amp;quot;}&lt;/code&gt;. &lt;code&gt;code&lt;/code&gt; hashes the script’s &lt;b&gt;lexed token stream without comments, spaces or repeated blank lines&lt;/b&gt;, and &lt;code&gt;run&lt;/code&gt; omits the benchmark (which changes no trade). Adding comments, adjusting spaces or blank lines, or re-running the exact same setting after adding new market days keeps the same try key; changing a rule, threshold or stop adds a new try.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Lineage events and conservative correlation.&lt;/b&gt; Deleting a run folder from disk never erases its tries from &lt;code&gt;tries.jsonl&lt;/code&gt;. Renaming or duplicating a script inside Rearview (or via &lt;code&gt;rvw tries&lt;/code&gt;) records &lt;code&gt;rename&lt;/code&gt; and &lt;code&gt;copy&lt;/code&gt; events so the script’s trial history follows it. Because &lt;code&gt;tries.jsonl&lt;/code&gt; stores only &lt;code&gt;[key, sharpe]&lt;/code&gt; rather than every historical daily return series, single-run &lt;code&gt;Every try&lt;/code&gt; DSR conservatively sets &lt;code&gt;N_eff = N&lt;/code&gt; (no correlation discount), whereas an in-memory search computes pairwise &lt;code&gt;ρ&lt;/code&gt; across its own trials.&lt;/li&gt;&lt;li&gt;&lt;b&gt;SHA-256 hash chaining.&lt;/b&gt; Every line in &lt;code&gt;tries.jsonl&lt;/code&gt; records a sequence number (&lt;code&gt;seq&lt;/code&gt;), UTC timestamp and &lt;code&gt;prev&lt;/code&gt;, the SHA-256 hex digest of the previous line. The Mac app stores the last seen head (&lt;code&gt;tries_head&lt;/code&gt;) in &lt;code&gt;rearview-project.json&lt;/code&gt;. Editing an earlier line breaks the &lt;code&gt;seq&lt;/code&gt;/&lt;code&gt;prev&lt;/code&gt; chain and permanently flags the log as &lt;code&gt;edited&lt;/code&gt;; deleting or truncating the file so &lt;code&gt;tries_head&lt;/code&gt; is missing causes the next append to write a permanent &lt;code&gt;gap&lt;/code&gt; event (&lt;code&gt;missing&lt;/code&gt; or &lt;code&gt;cut&lt;/code&gt;) and flag the count as potentially low from then on.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Sealed lockbox with embargo.&lt;/b&gt; Before starting an agent loop, you can seal the most recent years of data in a lockbox. Rearview suggests the smallest embargo gap needed by the project’s scripts (lookback plus holding period) so indicators cannot reach across the boundary. Every in-app run, search and &lt;code&gt;--project&lt;/code&gt; CLI invocation (or standalone CLI run given &lt;code&gt;--lockbox-start&lt;/code&gt;) stops short of the lockbox until you reveal it once at the end.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;Since Rearview 1.1, a run’s Deflated Sharpe counts every try on its data in the project, whichever script made it, so an agent that saves each variant as a new file adds to the same count rather than starting a fresh one. Before each try, the app shows the Sharpe that try needs to count, and the project can cap the tries the command line adds on its data: past the cap, the agent’s next &lt;code&gt;rvw … --project&lt;/code&gt; command is refused before it starts.&lt;/p&gt;&lt;h2 id=&quot;honest-limits&quot;&gt;Honest limits&lt;/h2&gt;&lt;p&gt;A tries log is a &lt;b&gt;research discipline aid, not security&lt;/b&gt;. Anyone with filesystem access can edit both files, though casual tampering shows. More importantly, Rearview is explicit about what it still cannot count:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Runs made outside the project (&lt;code&gt;rvw run --store ...&lt;/code&gt; without &lt;code&gt;--project&lt;/code&gt;) or in another backtester.&lt;/li&gt;&lt;li&gt;Ideas taken from papers or forums that were themselves selected from dozens of unpublished tests.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;For that reason, every Rearview verdict labels its trial count &lt;b&gt;&amp;quot;at least N trials&amp;quot;&lt;/b&gt;. Pairing an agent loop with &lt;code&gt;--project&lt;/code&gt; logging, a sealed lockbox, PBO and walk-forward tests lets you keep the speed of AI-assisted iteration without quietly losing track of how hard you searched.&lt;/p&gt;&lt;p&gt;Read the full specification in &lt;a href=&quot;/docs/research/&quot;&gt;Is this result luck?&lt;/a&gt;, or test your own trial count in the &lt;a href=&quot;/is-my-backtest-luck/&quot;&gt;Deflated Sharpe Ratio calculator&lt;/a&gt;.&lt;/p&gt;</content></entry>
<entry><title>Five silent reasons the same trading strategy gives different results across backtesters</title><id>https://rearview.page/writing/why-identical-strategies-diverge/</id><link rel="alternate" type="text/html" href="https://rearview.page/writing/why-identical-strategies-diverge/"/><published>2026-11-20T00:00:00Z</published><updated>2026-11-20T00:00:00Z</updated><summary>Signal-bar fills, pre-adjusted prices, survivorship and delisting haircuts, cash shortfalls, and tie-breaking and calendar conventions, plus a 5-step reconciliation protocol.</summary><content type="html">&lt;p&gt;Port a simple systematic strategy from one backtester to another and the results almost never match on the first try. Often the trade count is off by 15%, the drawdown shifts by several percentage points, and the Sharpe ratio moves enough to change your mind about trading it.&lt;/p&gt;&lt;p&gt;Most of the time, neither backtester has a bug. They simply made five default choices differently. Because those choices compound over thousands of trades, a small rule difference in year one produces a different portfolio by year five. (This note focuses on daily portfolio strategies; intraday bars add session boundaries, extended hours and start- vs. end-of-bar timestamp conventions as well.)&lt;/p&gt;&lt;p&gt;Here are the five mechanics that cause most divergences, with concrete places they show up.&lt;/p&gt;&lt;h2 id=&quot;1-signal-bar-fills-vs-next-session-fills&quot;&gt;1. Signal-bar fills vs. next-session fills&lt;/h2&gt;&lt;p&gt;Suppose your rule buys a stock when its 2-day RSI closes below 10.&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Same-bar close fill:&lt;/b&gt; A tool that defaults to filling on the signal bar executes at that exact close. That assumes you knew the final closing print, ranked every candidate across the universe, sized the orders from final closing equity and filled at that same close with zero gap.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Next-session open fill:&lt;/b&gt; An engine that evaluates rules at the close and fills at the next session’s open (&lt;code&gt;next_open&lt;/code&gt;) absorbs the overnight gap.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;For mean-reversion strategies in particular, the overnight gap between the signal close and the next morning’s open is often a large share of the trade’s return. Filling on the signal bar inflates backtest returns and produces orders you cannot actually place in an evening routine. (&lt;code&gt;bt&lt;/code&gt;, for example, fills at the decision close by default, whereas Rearview defaults to &lt;code&gt;fill next_open&lt;/code&gt; and marks &lt;code&gt;fill same_close&lt;/code&gt; runs as approximate.)&lt;/p&gt;&lt;h2 id=&quot;2-pre-adjusted-prices-vs-as-traded-prices&quot;&gt;2. Pre-adjusted prices vs. as-traded prices&lt;/h2&gt;&lt;p&gt;This is the most common source of distortion when importing free per-symbol CSV files.&lt;/p&gt;&lt;p&gt;Suppose a stock traded at $20 in 2016, later split 4-for-1 twice (16-for-1 total, pulling the split-adjusted 2016 price to $1.25), and paid roughly a 1.3% annual dividend yield over the next decade (about a &lt;code&gt;0.88&lt;/code&gt; multiplicative dividend factor). In a vendor’s back-adjusted CSV, that 2016 close shows up around &lt;b&gt;$1.10&lt;/b&gt;.&lt;/p&gt;&lt;p&gt;Scale-invariant indicators such as &lt;code&gt;rsi&lt;/code&gt;, &lt;code&gt;roc&lt;/code&gt; or &lt;code&gt;close &amp;gt; sma(close, 200)&lt;/code&gt; are unaffected by multiplicative split adjustment. Three other things break immediately:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Fixed-dollar price filters leak future outcomes.&lt;/b&gt; A rule requiring &lt;code&gt;close &amp;gt; $5&lt;/code&gt; skips stocks that were actually trading at $20 in 2016. Worse, backward split adjustment introduces subtle look-ahead into price filters: future big winners that forward-split get pushed below $5 historically (excluded), while future losers that reverse-split get pushed above $5 (included).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Share counts and per-share commissions distort.&lt;/b&gt; Buying $10,000 of a stock at $1.10 buys &lt;code&gt;floor(10,000 / 1.10) = 9,090&lt;/code&gt; shares instead of &lt;code&gt;500&lt;/code&gt; shares. At a &lt;code&gt;$0.005/share&lt;/code&gt; commission, you get charged &lt;code&gt;$45.45&lt;/code&gt; instead of &lt;code&gt;$2.50&lt;/code&gt;: an 18.2x commission error on a single entry.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Dollar-volume filters break if volume is not inverse-adjusted.&lt;/b&gt; Dividing price by 16 without multiplying volume by 16 shrinks historical dollar volume by 16x.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;To avoid all three, an engine needs to store &lt;b&gt;as-traded prices&lt;/b&gt; alongside separate split and dividend tables, and compute three distinct views using only past actions:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;As-traded prices&lt;/b&gt; for fills, price filters (&lt;code&gt;close &amp;gt; $5&lt;/code&gt;), share counts and per-share commissions.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Split-adjusted prices&lt;/b&gt; for price averages and levels (&lt;code&gt;sma&lt;/code&gt;, &lt;code&gt;highest&lt;/code&gt;, &lt;code&gt;atr&lt;/code&gt;) so a 2-for-1 split does not look like a 50% crash.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Total-return prices&lt;/b&gt; (splits plus reinvested dividends) for return-based and percentage indicators so an ex-dividend drop does not trigger a false dip buy.&lt;/li&gt;&lt;/ul&gt;&lt;h2 id=&quot;3-survivorship-bias-and-delisting-haircuts&quot;&gt;3. Survivorship bias and delisting haircuts&lt;/h2&gt;&lt;p&gt;Two backtests on &amp;quot;S&amp;amp;P 500 stocks from 2010 to 2025&amp;quot; are often testing different markets:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Today’s constituents vs. point-in-time membership.&lt;/b&gt; Running a rule on today’s 500 tickers guarantees every stock in your 2010 universe survived the next 15 years. Point-in-time membership only allows trading stocks that were actually in the index on each historical date.&lt;/li&gt;&lt;li&gt;&lt;b&gt;What happens when a stock delists.&lt;/b&gt; Even with delisted tickers in the database, engines disagree on the exit price. QuantConnect’s &lt;code&gt;LEAN&lt;/code&gt; closes every delisted position at its last traded price; Rearview closes an acquisition at its last close, and applies a configurable performance-delisting haircut (by default 30% on NYSE and 55% on Nasdaq, following the historical averages reported by Shumway (1997) and Shumway and Warther (1999), though real delisting losses vary widely around those averages). Some tools silently drop a stock whose CSV ends without booking an exit at all.&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;On mean-reversion strategies that deliberately buy sharp drops, delisted stocks are disproportionately selected as candidates. Leaving them out or closing them at par can turn a losing rule into a paper winner.&lt;/p&gt;&lt;h2 id=&quot;4-cash-shortfalls-order-sequence-and-commission-aware-sizing&quot;&gt;4. Cash shortfalls, order sequence and commission-aware sizing&lt;/h2&gt;&lt;p&gt;Suppose your portfolio holds 10 positions at 10% of equity each, and your signals are generated at Monday’s close. On Tuesday morning, two buys gap up 4%, leaving your account $380 short of buying the exact share counts computed from Monday’s close.&lt;/p&gt;&lt;p&gt;Backtesters handle that $380 shortfall in at least four different ways:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Shrink the share count&lt;/b&gt; to fit available cash (Rearview’s rule).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Skip the order entirely&lt;/b&gt; because buying power does not cover the requested shares (&lt;code&gt;LEAN&lt;/code&gt;’s default).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Allow cash to go negative&lt;/b&gt; on margin or an implicit overdraft (&lt;code&gt;bt&lt;/code&gt;’s default when target weights plus costs exceed cash).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Pre-shrink every order&lt;/b&gt; so the commission fits inside the target dollar amount (&lt;code&gt;bt&lt;/code&gt;’s commission-aware rebalancer).&lt;/li&gt;&lt;/ul&gt;&lt;p&gt;Order sequence matters just as much: whether sells execute before buys at the open, whether stops are live on the entry day itself, and whether a stock that gaps past a stop fills at the open or at the stop price.&lt;/p&gt;&lt;h2 id=&quot;5-tie-breaking-and-calendar-conventions&quot;&gt;5. Tie-breaking and calendar conventions&lt;/h2&gt;&lt;p&gt;Two subtle portfolio mechanics surprise even experienced quants:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Deterministic tie-breaking.&lt;/b&gt; Suppose your strategy holds at most 10 stocks, 3 slots are open, and 9 candidates have the exact same integer or rounded rank score (common with discrete filters, bar counts or short-window indicators). One engine breaks ties alphabetically by ticker; another by internal security ID; a third by hash-map iteration order (which can even change between runs). Giving the last slot to stock A instead of stock B changes Tuesday’s cash, which changes Wednesday’s sizing, which cascades across the rest of the decade.&lt;/li&gt;&lt;li&gt;&lt;b&gt;252 vs. 365 trading days.&lt;/b&gt; If your strategy holds coins (or mixes US stocks with 24/7 crypto), your equity curve has 365 daily returns a year rather than 252. Annualising Sharpe, Sortino and volatility with &lt;code&gt;sqrt(252)&lt;/code&gt; instead of &lt;code&gt;sqrt(365)&lt;/code&gt; understates Sharpe, Sortino and volatility by about 17% (&lt;code&gt;sqrt(252 / 365) ≈ 0.83&lt;/code&gt;; the 365-day figure is &lt;code&gt;sqrt(365 / 252) ≈ 1.20x&lt;/code&gt; larger). The same issue appears when combining two international exchanges (such as NYSE and ASX) whose combined calendar has more than 252 trading days a year.&lt;/li&gt;&lt;/ul&gt;&lt;h2 id=&quot;a-5-step-protocol-for-reconciling-two-backtesters&quot;&gt;A 5-step protocol for reconciling two backtesters&lt;/h2&gt;&lt;p&gt;When you move a strategy to a new engine and the headline numbers disagree, do not stare at CAGR or Sharpe. Reconcile from the first fill forward:&lt;/p&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Zero out costs, interest and margin.&lt;/b&gt; Set commissions, slippage and cash interest to zero in both tools first.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Match fill timing and sizing.&lt;/b&gt; Align &lt;code&gt;next_open&lt;/code&gt; vs. &lt;code&gt;same_close&lt;/code&gt;, whole vs. fractional shares, and confirm whether positions size from signal-day close equity.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Inspect the first diverging trade.&lt;/b&gt; Sort the trade lists chronologically and find the very first date where the two engines bought a different ticker, a different share count or filled at a different price.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Check that single day’s candidates and prices.&lt;/b&gt; Verify whether the difference came from price adjustment (split or dividend), universe membership, a tie in rank, or cash remaining after an overnight gap.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Re-enable costs and corporate actions one at a time.&lt;/b&gt; Once zero-cost trades match, turn commissions, slippage, margin and delisting haircuts back on so you know exactly which rule accounts for any remaining gap.&lt;/li&gt;&lt;/ol&gt;&lt;p&gt;For Rearview’s exact defaults and how to match AmiBroker, RealTest or Wealth-Lab step by step, see &lt;a href=&quot;/why-results-differ/&quot;&gt;Why your results differ from another backtester&lt;/a&gt; and &lt;a href=&quot;/docs/verification/&quot;&gt;How Rearview is checked&lt;/a&gt;.&lt;/p&gt;</content></entry>
</feed>
