Rearview Download

Checking a portfolio backtester against a clean-room second engine, vectorbt, bt and LEAN

November 2026 · Figures below are from the Rearview 1.0.2 release check (18 November 2026); the latest numbers live at How Rearview is checked.

Small unit tests on short arrays catch obvious coding mistakes, such as an off-by-one loop or a broken moving average, but they rarely catch what actually breaks a portfolio backtest: an overnight gap that exhausts cash mid-session, a reverse split colliding with a short position, a delisted stock that quietly disappears from a universe, or a subtle look-ahead leak in a cross-sectional rank.

When building Rearview, we wanted a stricter release bar: before every release, the engine has to prove it follows its own written rules, matches a separate implementation written without reading its source code, agrees with three open-source backtesters on the accounting and mechanics they share, and cannot see future prices across thousands of cut-date replays.

This note describes the four verification layers in our release CI, the recorded 1.0.2 figures, and the design differences we documented when anchoring Rearview against vectorbt, bt and QuantConnect’s LEAN.

1. Written rules and 104 hand-worked cases

Verification starts with a specification, not code. Rearview’s engine follows numbered written rules covering order fills, cash and share accounting, splits, dividends, delistings, Reg T margin, short borrowing, multi-strategy sleeves and shared cash pools, 365-day coin calendars, daily-settled futures chains with rolls, multi-exchange holidays, multi-currency conversion and intraday bars.

From those rules alone, we worked out 104 small scenarios by hand (80 single-strategy and 24 multi-strategy portfolio cases): every fill price, share count, cash balance, dividend, borrow fee and closing equity written down in exact integer millionths of a dollar ($0.000001). The Rust engine must reproduce every expected CSV file byte for byte.

Hand-worked cases catch an engine that disagrees with its specification, but they have two blind spots: nobody writes hand-worked cases for every strange interaction of features, and hand-worked cases cannot catch a bug in the rules themselves.

2. A second engine from the spec alone (2,200 random scenarios, 0 differences)

To catch bugs in the rare combinations nobody writes by hand, we built a second engine (refbt) in Python from the written rules and hand-worked cases alone, without reading the Rust engine’s source code. Both engines and the harness live in our internal release CI.

A second implementation written by the same team does not prove the written rules are right on their own, since both engines follow the same spec; that is what Layer 3 is for. What it catches reliably are translation mistakes, edge-case divergences and multi-feature interactions.

Before every release, the cross-check harness generates 2,200 random scenarios across five profiles:

  • Ties (every 10th scenario): many candidates with identical rank scores, exercising deterministic tie-breaking.
  • Stress (high leverage): margin calls, forced liquidations and account wipe-outs.
  • Portfolios: two or three strategies with independent signals, random weights, fixed sleeves (half with rebalancing that sells holdings to restore weights) or one shared pool of cash.
  • Lean: most optional features switched off, isolating core accounting.
  • Normal: random combinations of long and short entries, adds, scale-outs, stops, targets, time exits, corporate actions, late listings, delistings, settings changing by date and multiple markets.

The Rust CLI (rvw) runs on the imported binary store while refbt reads the raw CSV folder directly, so the check exercises the importer too. Every canonical output file (orders.csv, fills.csv, events.csv, trades.csv, daily.csv) is compared byte for byte. On Rearview 1.0.2, all 2,200 scenarios matched with 0 differences, and both engines agreed on all 368 invalid and boundary configurations (accepting and rejecting the exact same set).

The same differential check covers 760 live-account replays (45,301 broker fills, including partial fills, late sells, position reversals, deposits, withdrawals, coin fills and fractional shares), random-entry baselines, stress tests and performance attribution.

Byte-for-byte agreement is possible because cash and prices are stored in exact integer millionths of a dollar ($0.000001), never binary floating point, and parallel work across securities reduces in a fixed order. The same run produces the exact same bytes on one core or eighteen.

3. Anchoring against vectorbt, bt and QuantConnect LEAN (220 of 220)

Two engines written from the same specification can still share a mistake in the specification itself. To test our accounting and portfolio rules against outside implementations, we run 220 randomised scenarios through three independent open-source backtesters:

Outside engineWhat it anchorsHow it is drivenScenariosFills checked
vectorbt (1.1.1)Long entries, exits, costs, fractional shares, 365-day coin years, AUD account holding USD stocks, 5-minute intraday barsPre-shifted share orders (and pre-converted FX prices for AUD) to verify fill accounting, equity and Sharpe/Sortino80 of 806,644
bt (1.3.0)Weekly, monthly and quarterly rebalancing to equal or fixed weights, whole and fractional shares50 orders-mode fill replays plus 50 weights-mode runs where bt picks rebalance dates, ranks and shares independently100 of 10019,731
QuantConnect LEAN (18139)Short positions, 1.5x margin, forward/reverse splits, dividends and acquisition delistingsReplays Rearview’s entry and exit fills while LEAN independently applies splits, dividends, delistings, margin and cash40 of 40796

In bt, whole-share runs without costs show a maximum relative closing-equity difference of 1.97e-15 on orders and 9.89e-16 on weights (fractional-share weights stay within 8.06e-08). In vectorbt without costs, trades match exactly and Sharpe and Sortino ratios agree within 1e-6 relative (using 365 days for coin runs and 252 for stock runs). In LEAN, shares held, cash and equity agree after every close across 43 splits, 115 dividends and 18 delistings (within $0.000001 after a reverse split, where LEAN computes fractional cash-in-lieu through a binary float, and with equity alone compared on a delisting’s final close since LEAN books the close at the next open).

Stops, trailing stops, limit orders, multi-strategy portfolios, margin calls and performance-delisting haircuts are verified by refbt and hand-worked cases rather than these outside anchors.

Where the engines disagree by design

The most useful outcome of building external anchors was documenting the exact places where mature backtesters make different legitimate choices. Each of these has to be either aligned in the scenario config or left outside the anchor and documented explicitly:

  • Rebalancing on the last bar (bt): bt’s RunPeriod algos do not rebalance on the dataset’s final date unless run_on_last_date is set, whereas Rearview can emit rebalance orders on the last day.
  • Decision bar vs. next open (bt): bt fills at the same bar’s close on which it computes target weights. Rearview decides at a close and fills at the next session’s open by default, so the weights anchor explicitly selects same_close_approx while the orders anchor covers next-open fills.
  • Commission-aware sizing (bt): bt shrinks a buy (or enlarges a sell) so that the commission fits inside the target dollar allocation. Rearview sizes from the target weight alone and only shrinks a buy if cash runs short.
  • Negative cash vs. shrunk orders (bt and LEAN): When target buys slightly exceed available cash, bt lets cash go negative, LEAN rejects an order that exceeds buying power, and Rearview shrinks the order’s share count to fit available cash.
  • Short borrow fee timing (LEAN): LEAN starts charging a short’s borrow fee from the second day it observes the position, based on that day’s value; Rearview charges from the first night held, based on the previous close.
  • Delisting close timing and haircuts (LEAN): LEAN closes every delisted position at its last traded price at the start of the next day. Rearview closes at the delisting day’s close and applies a configurable haircut (default 30% on NYSE, 55% on Nasdaq) when the delisting was for poor performance rather than an acquisition.
  • Margin liquidation order (LEAN): On a margin breach, LEAN trims positions partially back to its maintenance threshold; Rearview liquidates whole positions, largest unrealised loss first, against FINRA-style maintenance rules.

None of these are bugs in bt or LEAN. They are design choices, and moving a strategy from one engine to another without accounting for them causes equity curves to diverge even on identical data.

4. Look-ahead replay, planted leaks and fuzzing

A backtest that peeks at tomorrow’s prices looks wonderful and is worthless. Before every release, Rearview tests the entire pipeline (split and dividend adjustment, point-in-time index membership, weekly and monthly bar aggregation, cross-sectional ranking, strategy evaluation and simulation) in three ways:

  1. Cut-date replay (10,924 cut-date runs across 71 scripts and 4 portfolios): History is truncated at cut dates landing on splits, dividends, delistings, index changes, month ends, late listings and inside intraday sessions (first bars, last bars and half days). Every decision and fill up to each cut must come out identical to the run on the full history (3,393,969 fills and 1,525,185 trades replayed on 1.0.2 with zero differences). Scans are replayed the same way across 23 cut dates and 9,880 rows.
  2. Future perturbation: Randomly altering every price after a cut date must not change a single decision before it.
  3. Planted leaks: To prove the detector actually works, the suite feeds intentionally leaky inputs (signals reading tomorrow’s close, adjustments computed from future splits, universes that know future delistings) and requires the test to catch every single one.

Finally, the CSV importer, converters and store reader are fuzzed with 200,000 broken inputs (flipped bytes, truncated files, shuffled rows, NaN, negative and huge numbers, renamed columns and damaged stores) with 0 crashes and 0 unhandled failures, and 104 golden replays check that saved runs still reproduce byte for byte across releases.

What this still cannot catch

Verification only matters if you state its boundaries plainly:

  • A rule that both Rearview engines and every outside engine get wrong in the same way. Only hand-worked cases and domain review guard against that.
  • A leak inside your own data. If you import today’s S&P 500 constituents or pre-adjusted vendor prices without corporate actions, the engine cannot invent missing delistings. Rearview’s import report flags what the files cannot tell it.
  • Markets not yet anchored to an outside engine. Futures rolls, forex carry and multi-exchange calendars are verified by hand-worked cases and the Python reference engine, not yet against a third-party engine.
  • Real broker nuances. The short-sale price test (Reg SHO Rule 201) is not modelled, and a broker can still book a corporate action or borrow rate differently from your market data.

The latest verification figures and every speed budget are published automatically from our release check at How Rearview is checked. If you ever find a result that looks wrong, Rearview’s Help ▸ Report a Wrong Result… bundles the script, settings and data fingerprint (without your bars) so we can replay it and add a new reference case.