Signal Performance — Read Backtests Honestly and Don't Be Misled

Every platform is eager to show you a beautiful chart where its strategy beats the market. QMA has its own tab for this — Performance / Signal Performance (/performance). But unlike marketing slides, it's built on a principle that the QMA team takes deadly seriously:

We don't promise. We show numbers — and teach you to read them critically, including where they have weaknesses.

---

What Performance Is For

Performance takes historical data (past signals and ratings) and retrospectively calculates how they would have turned out. It serves three purposes: proof instead of promises, selection of credible signals, calibration of expectations.

★ Insight: Performance doesn't say "buy this." It says "this is how this type of signal has historically performed — the decision is yours."

---

What Performance Displays

1. Rating-tier statistics — how each rating tier fared historically.

🚨 CORRECTION NOTE (31 July 2026) — the performance figures that used to stand here are WITHDRAWN.

>

They rested on a historical scores table whose older part was shown on 15 July 2026 to have been

derived from later prices — that is, from the future. On 31 July 2026 the significance

threshold was also tightened: it has to account for every test we have ever run, not for the

variants of a single run.

>

The figures are not being replaced by smaller ones. The inputs were never valid as of the day

in question, so an honest replacement does not exist — the claim goes away entirely. Only

forward tracking since 18 February 2026 is trustworthy.

2. Strategy Backtests — win rate, average return, profit factor, MFE/MAE, top trades, alpha vs SPY.

3. Comparison with the Market (SPY) — a strategy with +20% looks great until you find out the index made +25% → underperformed. This difference is called alpha.

---

How to Read a Backtest HONESTLY

Trap No. 1: Win rate without average return means nothing

	Strategy A	Strategy B
Win rate	90 %	40 %
Avg. gain	+2 %	+25 %
Avg. loss	−20 %	−7 %
Result	loss-making	profitable

Strategy A wins 9 out of 10 and yet ruins — one loss eats all the small gains.

★ Insight: Win rate tells how often you're right. Average return tells how much it earns. The latter is more important.

Profit factor

PF = sum of gains ÷ sum of losses. 1.0 = zero; <1 loss-making; 1.5–2.0 solid; >3.0 excellent (but beware of over-fitting). PF 2.0 means that for every 1 unit lost, the wins earned 2.

MFE / MAE

MFE = maximum gain a trade saw. MAE = deepest loss. High MAE = trades go deep into loss (psychologically demanding, risk of being stopped out).

Statistical significance — the t-statistic, the sample and the number of trials

A win rate around 80 % over thirteen trades is not proof, it is an anecdote. Flip a coin thirteen times and sometimes ten heads come up.

There is a second, less familiar trap: how many hypotheses were tested before this one worked.
The textbook "t above 2" applies to a single pre-registered test. Pick the best of hundreds of
variants and the bar has to rise with the number of trials — otherwise you read the result of a
search as a finding.

Label	Criterion	What it means
🟢 Statistically supported	t-statistic above a threshold derived from the total number of trials and a sufficient sample	Probably not a coincidence
🟡 Exploratory	anything below that	Interesting, unproven

After the threshold was tightened on 31 July 2026, only a handful of internal findings with very large observation counts remained in the green tier; the long-term "legends" are not among them.

★ Insight: The yellow "exploratory" badge is honesty, not shame. Relabelling a hypothesis as a

fact costs the reader money.

---

Why a Backtest is NEVER a Promise of the Future

1. Over-fitting — tuning to past noise, fails on new data. Symptom: suspiciously high PF. QMA tests on a split sample.
2. Idealized execution — reality: spread, fees, slippage.
3. Survivorship bias — bankrupt companies have disappeared from the data.
4. Regime change — a bull market strategy may fail in a correction.

★ Insight: Realistic rule — expect live alpha to be materially lower than the backtested one, often a fraction of it. And if the backtest read data that was not publicly known on that day, it does not shrink — it collapses entirely.

---

How to Use Performance

1. Open /performance.
2. Filter by statistical significance — start with only green.
3. Read the quartet together — win rate + average return + profit factor + n.
4. Check alpha vs SPY — doesn't add over the index? Buy an ETF on SPY.
5. Look at MFE/MAE (what fluctuations can you withstand).
6. Lower expectations well below the backtested level.

---

Connection: Performance → Strategy → Predictor

• Performance = past ("how has the signal historically performed?").
• Strategy = catalog of approaches (✅/⚠️/❌), where you choose the methodology.
• Predictor / live tracking = present ("what now, and does it hold live?").

Live results gradually supplement history → each strategy gains a higher n over time.

---

Tips

1. Read four numbers together.
2. Profit factor is your best friend (below 1.5 caution, above 3.0 suspicion of over-fitting).
3. < 30 trades = hypothesis.
4. Expect 30–50% of the backtest.
5. Alpha vs SPY is a test of sense.

---

Common Mistakes

Believing a high WR without context · ignoring n · taking a backtest as a promise · looking at returns without SPY · chasing the highest PF (over-fitting).

---

Disclaimer

QMA is an analytical and educational tool, not investment advice. Past performance is not indicative of future results. The numbers are based on historical backtests with inherent limitations (idealized execution, over-fitting, survivorship bias, regime change) and real results are usually lower. It is not a buy/sell recommendation. Decisions are yours; consider consulting a licensed advisor. Trading carries the risk of loss.