Each idea is written down — thesis, parameters, pass/fail rule — before any
data is looked at, then run exactly once against a frozen data snapshot. These are the
gates, in order. Two of them are marked legacy: they are kept because early verdicts were
decided with them, and because they are exactly what fooled those verdicts.
Entry test — obligation
Is someone actually obliged to trade badly here, and unable to stop?
Before any data work: name the counterparty, quote the sentence in a primary document (statute, rule, prospectus, filing) that forces the trade, and say why it cannot stop. No quote, no test.
Most "forced flow" stories dissolve when you read the obliged party's own document — it usually reserved a way out. The cheapest predictor found so far is who wrote the rule — a regulator binds; a party writing its own document rarely does.
Pre-registration
Was the hypothesis, its parameters and its pass/fail rule written down before any data was looked at?
Thesis, signal, universe, exits and thresholds committed to git before the single run. One run. No variants afterwards.
The textbook overfitting loop is test → look → adjust → call it validation. Pre-registration is the only known antidote.
Mechanism precondition
Does the structure the strategy relies on exist in the data at all?
A pre-registered structural check run before any backtest (e.g. cointegration for pairs). If it fails, the strategy is never simulated.
Testing returns of a mechanism that is not there only produces noise to overfit.
Raw significance (legacy)
Are the strategy's raw returns statistically different from zero?
t-stat of raw per-trade returns above a Bonferroni/HLZ-adjusted threshold for the number of pre-registered variants.
Used until May 2026. Kept on the site because several early verdicts were decided with it — and it is exactly what fooled them.
Market-adjusted alpha (legacy)
Does the strategy beat SPY over the same holding windows, assuming beta = 1?
alpha_trade = pnl − direction × spy_pnl; t-stat above threshold.
Introduced 2026-05-23 after a long-only strategy passed every formal gate on bull-market drift. Superseded by Jensen alpha in Sept 2026 because assuming beta = 1 mis-sizes alpha whenever real beta is far from 1.
Gate 7 — Jensen alpha
After regressing out market exposure, is there alpha left?
Intercept of OLS(trade return ~ SPY return over the same window), t-stat ≥ 1.65 one-sided (Bonferroni-raised when the strategy belongs to a tested family). Since 2026-10-30 the standard error is clustered by calendar date (Driscoll-Kraay); before, it was HAC(10) over trade rows, which on a null panel rejected 21% of the time at a nominal 5%.
Most ideas this lab measured died here (the count is on the home page). Most "edges" are beta that a bull market paid for.
Gate 8 — Capture-preserved alpha
Does the subset of trades a real portfolio can actually take keep the alpha?
Alpha re-measured on the trades a slot-limited portfolio simulation executes, at the declared sizing. t_captured ≥ 1.65·√(n_captured/n_engine) AND capture ratio (mean alpha captured / mean alpha engine) ≥ 0.50.
A signal can have real alpha across 10,000 trades while the 800 a 10-slot portfolio takes have none. The ratio leg punishes dilution; the t leg punishes noise.
Gate F — Friction
Is the alpha per trade large enough to pay for its own execution?
Mean Jensen alpha per trade ≥ 0.30% (3× the generic 0.10% round-trip slippage).
Statistically real and economically useless are different verdicts. No amount of capital fixes slippage that exceeds the edge.
Stability
Is the alpha present in both halves of history?
Alpha t > 0 in trades entered 2014-2019 AND in 2020-2026.
Catches an "edge" that is one regime (e.g. 2020-21) averaged over twelve years.
Own alpha
Is the alpha new, or already captured by a strategy that passed?
Jensen t on the trades that do NOT overlap with an existing validated strategy ≥ Bonferroni threshold.
Three archived shorts turned out to have real alpha — all of it inside the trades of the one strategy that survives.