How to Read a Backtest — t-Statistic, Win Rate and Alpha vs the Index, in Plain Words

🚨 CORRECTION NOTE (31 July 2026) — the performance figures that used to stand here are WITHDRAWN.

>

They rested on a historical scores table whose older part was shown on 15 July 2026 to have been

derived from later prices — that is, from the future. On 31 July 2026 the significance

threshold was also tightened: it has to account for every test we have ever run, not for the

variants of a single run.

>

The figures are not being replaced by smaller ones. The inputs were never valid as of the day

in question, so an honest replacement does not exist — the claim goes away entirely. Only

forward tracking since 18 February 2026 is trustworthy.

When someone says "my methodology does 50 % a year", there are three questions you have to ask —
otherwise you end up trusting something that never happened.

This article teaches you those questions, for any backtest, ours included.

Question 1: Win rate — how often was it right?

What it is: the share of periods in which the methodology beat its benchmark.

The trap: win rate says nothing about magnitude. Something that wins ten times by 1 % and
loses once by 50 % has a 91 % win rate and still loses money. A high percentage on its own is nothing.

How to read it properly: a win rate only means something together with the average return per
trade. If one of the two numbers is missing, it is missing on purpose.

Question 2: Alpha vs the index — how much above the market?

What it is: the gap between the methodology's return and the index (S&P 500 / SPY) over the same
window.

Example: methodology +30 %, index +20 % → alpha = +10 percentage points (pp).

Why "30 % is nice" is not enough: when someone tells you "I made 30 %", ask *"and what did the
index do over the same period?"*. If the index did 28 %, that person got paid for adding almost
nothing. Active stock selection only makes sense if it beats a cheap index after costs.

Question 3: t-statistic — was it luck?

This is where statistics begins, and where most marketing falls apart.

What it is: the t-statistic measures how likely the result was chance rather than a real edge.

And now the part almost everyone forgets — the number of attempts. Flip a coin ten times and six
heads surprises nobody. Flip it a thousand times and pick the best ten-flip stretch — that is no
longer a finding, it is the result of searching. The significance threshold therefore has to rise
with how many hypotheses you have ever tested. The classic "t above 2 is enough" applies to a
single pre-registered test, not to the best result out of several hundred runs.

Since 31 July 2026 QMA keeps a permanent ledger of trials and derives the threshold from it. In
practice the bar sits well above that textbook 2 — and a number of previously "verified" results did
not clear it. Which ones, and why, is set out in the correction note above.

Realistic expectations — why the headline number never materialises

Headline alpha is uncorrected. In reality you have to subtract:

1. Survivorship bias — the backtest works with companies that exist today. Those that went

bankrupt or were delisted are not in the data. The real universe was worse.

1. Slippage — the price the backtest "bought at" is not the price you get.
2. Costs — commissions, taxes, currency conversion.
3. Look-ahead bias — the backtest sometimes sees a figure before it was public. This is the

nastiest trap: the only way to catch it is to ask what exactly was truly known on that day.

★ A lesson we paid for: look-ahead hides behind honest-looking tables too. Our historical

scores looked like a record of the past while being derived from later prices. That is why every

figure now gets asked where the input came from before anyone looks at the result.

Forward tracking is the one test that cannot be gamed

A backtest can be dressed up; forward-measured data has nowhere to hide. That is why backtested and
live figures are never merged into one row here, and the live figure always wins.

Takeaway — five questions for any backtest

1. Is the average return per trade shown next to the win rate? A win rate alone means nothing.
2. Is alpha vs the index in percentage points, not percent? If not, it is usually a trick.
3. Are the t-statistic and the number of trades stated? If not, ask for them.
4. How many hypotheses were tested before this one worked? Without that answer significance

cannot be established.

1. Is there forward measurement? The past can be tuned; the present cannot.

---

This > text describes how performance statistics are read — it is not an instruction to buy or sell. > Past performance does not guarantee future results. Investing carries the risk of capital loss.