Measurements · 11 findings

What we measured, and what we retired because of it.

A backtest you are shown is usually a survivor: the ones that did not work were never published. These are all of ours, including the misses and the tests that cost us features.

Each finding below was produced by an in-sample measurement in this codebase: measured on the same history the rules were written against, not on held-out data. Where a result is negative, it is stated as a negative. Where a result is unmeasured, it is stated as unmeasured rather than quietly implied to be good, those are different claims and conflating them is how this category misleads people.

We have deleted working code over these numbers: a scanner's directional claim, a circular exhaustion claim, and an “edge” claim on our own marketing page. The text below is rendered directly from the product's explainer module, so what you read here is what a paying subscriber reads inside the product.

Could this just be luck?

For each signal’s historical record we resampled the outcomes 2,000 times and asked whether the 95% range of its hit rate minus the base rate (the same outcome on matched signal-free bars) stays clear of zero. Events that cluster in time are counted at their effective sample size, so a burst of alerts in one sell-off is not treated as many independent wins. These are past, in-sample records (see methodology); a verdict says how unlikely the record is to be luck, not what happens next.

SignalEventsHit rate vs base95% range of the differenceVerdict
Quantitative Exhaustion: PB1 level reached
price reaches the PB1 level within the hunt window
47743.2% vs 35.7%+0.1 to +14.9 ptsAbove base rate
Confluence score at the 0.20 entry gate (rebuilt, 6 of 10 inputs)
price moves in the scored direction first, 5 sessions
32755.0% vs 57.2%-10.1 to +6.4 ptsNo verdict yet
about 4,307 more events at this effect size
First re-test of a swing level (price-only stand-in for nodes)
price reacts 0.30% away from the level within 2 sessions
1,47867.4% vs 48.1%+16.3 to +22.3 ptsAbove base rate
Scanner: exhaustion
price moves in the signalled direction first, 20 sessions
1,16152.1% vs 56.4%-9.3 to +0.8 ptsNo verdict yet
about 466 more events at this effect size
Scanner: fib proximity
price moves in the signalled direction first, 20 sessions
25,78149.9% vs 50.2%-0.9 to +0.4 ptsNo verdict yet
The observed difference is under half a point; no realistic sample would separate it.
Scanner: rsi divergence
price moves in the signalled direction first, 20 sessions
3,03948.5% vs 46.9%-1.0 to +4.1 ptsNo verdict yet
about 4,955 more events at this effect size
Scanner: volume spike
price moves in the signalled direction first, 20 sessions
5,16849.9% vs 51.2%-3.0 to +0.4 ptsNo verdict yet
about 4,422 more events at this effect size
Scanner: ma crossover
price moves in the signalled direction first, 20 sessions
2,01750.7% vs 50.3%-2.1 to +3.1 ptsNo verdict yet
The observed difference is under half a point; no realistic sample would separate it.
Scanner: relative strength
price moves in the signalled direction first, 20 sessions
14,00450.3% vs 49.9%-0.7 to +1.4 ptsNo verdict yet
The observed difference is under half a point; no realistic sample would separate it.
Scanner: composite score
direction of the composite card, 20 sessions
16,68950.0% vs 50.7%-1.5 to +0.1 ptsNo verdict yet
about 2,363 more events at this effect size

Dropped, results kept

Strategy or claimEventsHit rate vs base95% range of the differenceVerdict
PB1 "52-58% hit rate" sentence (2020-2026 window)
PB1 reached by session 8
12532.8% vs 22.8%-2.8 to +23.2 ptsNo verdict yet
about 85 more events at this effect size
Why dropped: The published sentence overstated the rate; it was retired and replaced with the measurement.
Long after a failed Quantitative Exhaustion
target reached within 40 sessions
4148.8% vs 52.8%-18.6 to +10.6 ptsNo verdict yet
about 553 more events at this effect size
Why dropped: Built, measured with no edge, and not shipped as a setup.

Not tested, and why

01 · confluence.score
Measured

The confluence score has been measured

The score has now been scored, in-sample, using the shipped compute_confluence_score over 34,297 bars and ten years. AT THE LIVE BOT'S OWN GATE (|score| >= 0.20, confidence >= 45) AND ITS OWN five-session hold, it carries no directional information the measurement can establish. On the bot tier (^GSPC): 327 onset-deduplicated entries, 55.0% correct against an instrument-and-month matched base of 57.2%, lift -2.1 pp, interval [-6.7, +2.1] pp, which straddles. Against a second, unstratified base of 53.8% the lift is +1.2 pp [-5.6, +7.3], also straddling. The wider index and wide tiers land the same way, and NOT ONE of the 4,113 cells in the shipped table survives the three-hurdle verdict gate. Two caveats travel with every number. First, only six of the score's ten components have ten years of history here, 62% of the shipped weight, so this is a result about that 62%, and any information in the live score would have to come from the option-derived remainder. Second, the same measurement run on a bar that is NOT an entry, carrying the entry's own direction claim, scores comparably, which is why nothing here is published as an inversion either.

02 · nodes.tap-decay
Measured

Tap decay · the 80 / 66 / 33 ladder

The framework this product competes with states that a first tap of a major node reacts about 80% of the time, a second 66%, a third 33%. It states it six times across its site, once as "not theory, it's what the data shows", and never once with a sample size, a date range, an instrument or a definition of "reaction". Those three numbers are also in this codebase, in setup.TAP_HOLD, where they help decide whether an entry is refused. So we measured what can be measured.

Over 11,792 touches of confirmed swing levels on SPY, QQQ, IWM and DIA across roughly 180 calendar months, reaction by touch number is FLAT: 67.4% on the first tap, 66.9% on the second, 69.3% on the third, 67.9% on the fourth and later. The collapse at the third touch does not appear anywhere - the third tap is the highest bucket measured.

The levels themselves are real, and that is the other half of the finding. Every bucket clears a ticker-, month- and direction-matched null by +15 to +19 pp, with month-block intervals that exclude zero: +19.3 pp on the first tap, +15.1 on the second, +18.8 on the third, +15.7 on the fourth and later. Levels work. It is the DECAY LADDER that does not survive, not the idea of a level.

This does NOT measure the claim as stated, and the distinction matters. The claim is about dealer gamma nodes. Rebuilding a historical options book needs open interest and implied volatility per contract per date, and the chain endpoint serves those live only - so the node version of this question cannot be asked of the past at all. node_history has been recording the book since it shipped and needs 120 sessions before it can answer directly.

What can be measured today is the claim's PRICE-ONLY SHADOW. Take a level with real structural memory - a confirmed swing pivot - and ask whether reaction falls with each subsequent touch. If repeated touches wear a level out, it should be visible there too, because the claim is about price behaviour at a remembered level and only incidentally about why the level is remembered.

A level is a bar whose high is the highest of the five sessions either side, or whose low the lowest. It cannot be known until those five sessions have closed, so touches are only counted from the confirmation bar onward - counting one inside that window measures hindsight, and hindsight is what produces a clean, false decay curve. A touch is a session trading within 0.15% of the level, with a three-session cooldown so a week spent loitering is one touch and not five. A reaction is price moving 0.30% away from the level, in the direction it approached FROM, within two sessions: an approach from above that keeps falling is a break, not a reaction. Same definition and same constants as the node harness, so the two stay comparable when that sample matures.

The null is the same test, in the same direction, on bars that touched no live level, matched on ticker AND calendar month AND direction. Each of those three matters on its own: pooling tickers judges a QQQ touch against an SPY yardstick, pooling months scores the market's own regime as the level's skill, and pooling direction compares a one-sided rate against a two-sided one, which in a market with drift manufactures a lift out of nothing. The interval is a calendar-month block bootstrap of the LIFT with both arms drawn on the same months, so the regime common to both cancels.

A NEGATIVE RESULT IS THE ONE A BADLY CHOSEN CONSTANT MANUFACTURES, so every parameter is swept: pivot lookback 2 to 20 (which moves the sample from 28,811 touches down to 2,300), touch band 0.05% to 0.30%, reaction 0.15% to 0.60%, horizon one to five sessions, cooldown one to ten. The absolute rate moves as those move - from 56% to 81% depending on how far and how fast price has to travel. The SHAPE does not. Across all 20 variants the largest fall from the first tap to the third is 1.15 pp, against the 47 pp the doctrine asserts. Not one variant shows a drop of even ten points.

The sweep also caught its own bug, which is worth recording because it is the quietest way to fake a robustness check: pivot lookback first returned three identical rows, because the constant was bound as a default argument at import. The sweep was testing one value three times and reporting it as robustness. It is resolved at call time now and a test fails if that regresses.

WHAT THIS CHANGES HERE. Not the constants: setup.TAP_HOLD still carries 80 / 66 / 33, because the price-only shadow cannot refute a claim about gamma nodes and replacing one unmeasured number with another would be the same mistake in the other direction. What changes is that the line which REFUSES a trade now reads "unmeasured doctrine" instead of a bare percentage, and the open question is narrower than it was: not "does reaction decay by tap", which has an answer for price levels and the answer is no, but whether dealer gamma nodes decay in a way price levels demonstrably do not.

03 · exhaustion.evidence
Measured

How the Buy Zone was measured

The question a Buy Zone answers is "does price come down here". So the measured outcome is exactly that: did the low of the window trade at or through the level. It is the same test detect_exhaustion_alerts already runs when it sets pb1_hit.

A rate on its own says nothing, because a shallow pullback from a two-day high is an ordinary thing for an index to do. So every rate is paired with the identical construction on bars belonging to no trigger streak, in the same instrument and the same month. Matching the instrument matters, because a trigger needs two sessions under 0.60% range and therefore fires on the quietest names, and a null pooled across everything would be a yardstick from a different market.

A rate and its base rate must describe the SAME events, or the difference between them is not a lift. An event whose instrument and month hold no signal-free bar has no base rate, so it leaves the cell entirely rather than counting toward the rate alone. The count of those is published on every cell as unmatched, and it is zero for every figure quoted here.

Three controls decide whether a lift is real. Intervals come from a calendar-month block bootstrap, because triggers arrive in clusters and on several instruments at once. The lift and its null are resampled TOGETHER, so the base rate is not treated as an exact constant. And the whole measurement is repeated on bars 35 and 70 sessions either side of each trigger, with each placebo removed from its own comparison group so it is not scored partly against itself: if those placebo bars showed the same lift, the finding would be about the era, not the trigger. They show roughly zero.

Every window is published, from day 8 out to day 60, so no window can be chosen after seeing which one flattered the model. So is the one specification choice that moved a headline number by more than a point: whether the signal-free arm keeps bars whose next five sessions contain a streak. Keeping them is the shipped and more conservative reading; dropping them reports +14.7 pp instead of +7.5 pp, and both ship under null_specification. Re-run the lot with python3 -m heatseeker.data.exhaustion_outcomes.

04 · exhaustion.validation-evidence
Measured

The day 8-9 close does carry information

A close below the trigger day's close on day 8 or day 9 happened on 44.2% of all 480 triggers (this test needs only nine forward sessions, so none is censored) against 30.1% on matched signal-free bars: a lift of +14.0 pp, interval +8.9 to +19.5 pp. Of everything the Q.E. model asserts, this and the shallow PB1 band are the two that separate from chance.

05 · exhaustion.arrival-evidence
Measured

The 16-19 window is not where the low lands

Of the 477 triggers whose day 8-40 hunt completed, the deepest low landed inside days 16-19 11.5% of the time, against 10.6% on matched signal-free bars, a lift of +0.9 pp with an interval of -3.4 to +5.1 pp. Four sessions out of thirty-three is about 12% by chance alone. Treat the band as a calendar marker, not a forecast.

06 · exhaustion.regime-evidence
Measured

The VIX buckets are not separated by the data

CALM (n=368) shows a PB1 lift of +8.5 pp on an interval of +3.5 to +12.4 pp. NORMAL (n=98) shows +5.2 pp on -3.9 to +13.6 pp, and its own interval contains zero, so at that sample size NORMAL is not distinguishable from no effect at all, which is a different statement from "a smaller effect". The two cannot be told apart either, which is why the CALM/NORMAL split in the target table is unsupported. ELEVATED has 11 triggers in ten years, below the 20 this platform will report on. CRISIS has zero: a trigger needs two quiet sessions, and VIX above 30 does not produce them. The CRISIS row of the target table is inherited, never fitted.

07 · exhaustion.deep-evidence
Measured

The Deep tier is untested, not conservative

The -16.18% extension was reached on 1 of 477 triggers inside the day 8-19 window in ten years (8 of 477 by day 40), no more often than on bars with no trigger at all. It is a drawn extreme, not a measured expectation.

08 · exhaustion.lag
Measured

The alert is dated one session late

A trigger is dated on the LAST zero-succession day of its streak, and whether today is the last one is not known until tomorrow closes: if tomorrow is also quiet the streak extends and the date moves. So a Q.E. alert is a one-session-lagged label, not a same-day signal. Every measured number on this page starts its forward walk at day 6, well past that, so nothing here depends on the difference.

09 · exhaustion.pb1
Measured

PB1 · Fib 0.236

Fib 0.236 × 10 = 2.36% below the Day 4-5 pivot high, identical in every VIX regime. That much is the framework's definition. What follows is measurement.

Two levels, not one, and the copy has to say which. The detector scores pb1_hit against the 2.124% variance line (PB1 widened by the prep multiplier) so that is the cell the headline rate belongs to. Over the 477 triggers whose day 8-19 window resolved, out of 480 fired across SPX, SPY, QQQ and IWM from 2011 to 2026, price traded into it 43.2% of the time against 35.7% on bars where no trigger fired, same instrument, same calendar month. Lift +7.5 pp, 95% month-block interval +3.5 to +11.2 pp. At the 2.36% line the ladder actually draws, the same test reads 38.4% against 31.6%, lift +6.8 pp, interval +2.5 to +11.0 pp. Both clear their base; neither is the other.

The trigger is a COMPRESSION event, so the obvious alternative reading is that price simply moves further afterwards in both directions. The test for that is the mirror: the identical construction run UPWARD off the Day 4-5 pivot low. The gap between the two (`+11.1 pp`, interval `+4.0 to +18.0 pp`) is the directional finding, and it is the figure to read. The raw upside lift of -3.6 pp is NOT a standalone finding: the two tests hang off opposite ends of the Day 4-5 spread, and a trigger's spread is 1.22% against 2.13% on a signal-free bar, which makes BOTH of its tests mechanically harder. That headwind cancels in the difference and does not cancel in either raw lift, where it works AGAINST the downside finding, so the +7.5 pp is if anything conservative.

The control that matters most: repeating the whole measurement on bars 35 and 70 sessions either side of each trigger (same instruments, same era, no trigger) returns lifts of +2.7, +0.7, +0.2 and +0.8 pp, every interval spanning zero. What is being measured is the trigger, not the period.

What this does NOT say. It does not say the band is the bottom; reaching a level and the level holding are different claims and only the first is measured. It does not say 43% is a forecast: on the 2020-2026 window alone the same cell reads 54.1% and its lift interval -1.0 to +17.8 pp contains zero, which is exactly why a bare rate must never be quoted without its base. And 477 triggers is not 477 independent events: SPX and SPY are one index, and 42.5% of the triggers are the same day counted twice.

10 · exhaustion.pb2
Measured

PB2 · the two deep lines

PB2 is drawn twice: the primary line is the p90 stretch-bottom fitted per VIX regime, the secondary is the Fib-pure 6.18% (0.618 × 10). Showing both was meant to keep the empirical and the harmonic readings side by side.

Measured over 477 triggers, 2011-2026, in the detector's own day 8-40 hunt window: the 6.18% secondary was reached 15.5% of the time against 12.2% on matched signal-free bars, lift +3.3 pp, interval +0.8 to +6.3 pp. That interval clears zero, so it is a separation, but a small one that nearly touches zero and is one cell among the many this table publishes; it is not the kind of margin the PB1 rung carries. The 7.86% CALM primary reads +1.7 pp on -0.3 to +3.7 pp and the 10.00% primary -0.5 pp on -1.5 to +0.6 pp. Both intervals contain zero: no measured edge.

In the tighter day 8-19 window (the one the ladder is hunted in first) none of the three separates at all: the 6.18% line reads +0.2 pp on an interval of -1.2 to +1.7 pp.

So the honest reading of the deep ladder is mostly descriptive: it shows how far a selloff would have to run to reach each Fibonacci rung. Only the Fib-pure 6.18% line, and only over the full day 8-40 hunt, does better than its own base, and the measured margin there is a few points, not a forecast.

11 · exhaustion.invalidation-long
Measured

The invalidation long · the setup that is usually not there

A Quantitative Exhaustion validates when price CLOSES BELOW the validation level inside the day 8-9 window. When it does not - the cycle that missed by 0.77 cents at the close is the canonical example - the framework reads the failure as its mirror: a long, entered at the level price refused to break, stopped under the low the cycle made getting there, targeting the swing high the ladder was measured down from.

The first result is about availability, not accuracy. Of 276 invalidated cycles across SPY, QQQ, IWM and DIA over fifteen years, 235 had already closed above the swing high by the time validation failed. There is no setup in those: the target is behind price. This is obvious in hindsight and was not obvious in advance - an invalidated cycle is one where the selloff never arrived, so the structure it would recover to is already spent.

In the 41 cycles where the target was still ahead, the setup reached it before its stop 48.8% of the time. The matched null - the same percentage target and stop, placed on the same ticker in the same calendar month with no cycle running - came back 52.8%. The lift is -4.0 pp with a 95% interval of -17.3 to +8.2 pp. It contains zero, and the point estimate is on the wrong side of it. Across eight combinations of horizon (20, 30, 40, 60 sessions) and same-bar tie-break, every single lift was negative.

The most useful thing this measurement produced was a trap it nearly fell into. Scored from the validation LEVEL, the setup reads 91.5% against a 72.7% null - a +18.8 pp edge with an interval nowhere near zero. That number is an artefact. Validation fails precisely BECAUSE price closed above the level, so a setup priced from the level starts with the move already partly banked: the target is nearer than its stated percentage and the stop further away, while the null starts from a standing position. Measuring from the validation close - the price a reader could actually get - removes the head start, and the edge goes with it.

One more distinction the surface has to keep. A cycle's target is its swing high, a price the market has ALREADY printed, so the target can never be above the record high - at most it can BE the record high, and only then is reaching it the same event as printing a new one. Of the setups that reached target, only about one in ten saw a new record high before the position closed. "Target reached" and "new all-time high" are different claims and this product will not quote one as the other.

The levels are arithmetic. The claims are what we keep honest.

Dealer gamma, exhaustion ladders and the succession grid compute what they compute. What this page is for is the line between that and what anyone can prove it predicts.

Create a free account Get the daily brief