Method
Pre-registered: thresholds are written here BEFORE results are seen, and every change is a dated entry at the end. A grade is never re-done because a result was disliked. The code in nocap/checks.py implements exactly this page; METHOD_VERSION there must match the version below.
Method version: 2026-11-03.4
1. Evidence (in force from 2026-11-03)
See DISCLAIMER.md. Captures keep the exact bytes, URLs, HTTP status and dating headers, UTC capture time and SHA-256; every capture, upload and quote is a hash-chained ledger event; the head is anchored daily (OpenTimestamps). A quote is accepted only when its words (after Unicode NFKC, curly-quote/dash/space normalisation and case folding) appear in the stored text of the capture. Every file we grade is first recorded as an upload, and the report cites it.
2. What we read
MT5 trade-history and strategy-tester reports (HTML, UTF-16 or UTF-8: the Positions or Deals table), MT4 detailed statements (Closed Transactions) and MT4 strategy-tester reports, and CSV with named columns. A trade's net result is profit + commission + swap (+ fee). Balance, deposit and credit rows are cash flows, not trades. Cancelled and pending orders are ignored. Anything we can't read is reported as unread, never guessed.
3. Enough evidence?
"Not enough evidence" (no letter grade) when any of these hold:
- fewer than 100 closed trades for a backtest, or fewer than 50 for a live record;
- the trades span less than 6 months (backtest) or 3 months (live);
- the file could not be read, or its trades conflict (see 4.1 FAIL items) so badly that no figure can be trusted.
4. The checks
Each check returns PASS, FLAG or FAIL (or N/A when the file doesn't carry what it needs), with the numbers behind it.
4.1 Genuine
- FAIL: a running balance column that doesn't follow from the trades (difference over 0.01 in account currency, after allowing for cash flows); the same ticket twice; a trade closing before it opens.
- FAIL: the report's own stated profit factor differs from the one recomputed from its trades by more than 10%. (MT4 states it on profit alone and MT5 after commission and swap, so both are recomputed and the closer one is compared; a difference within the report's own 2-decimal rounding counts as equal.)
- FLAG: stated profit factor differs by more than 2%.
- FLAG (backtests): modelling "Open prices only", or MT4 modelling quality under 90%, or MT5 history quality under 90%.
4.2 Survives costs
- Backtests: every trade is charged an extra realistic round-trip cost (spread plus slippage) from the cost table (
costs/provisional.json, marked provisional until replaced by aggregated measured spread curves), converted to money with the symbol's value per price unit worked out from the file's own trades. Live records: costs are already real, so nothing is added. - Profit factor after costs: PASS ≥ 1.30 · FLAG 1.00–1.30 · FAIL < 1.00 (or net result ≤ 0).
- Reported alongside: costs as a share of gross profit.
4.3 Hidden risk
- Martingale: share of trades whose volume is over 1.5× the previous trade's after that trade lost. FAIL > 10% · FLAG > 3%.
- Averaging down / grid: a trade opened on the same symbol and side while an earlier one is still open, at a worse price. Share of trades: FAIL > 20% with a stack of 4 or more open together · FLAG > 5% or a stack of 3.
- Worst single loss as a share of the balance before it: FAIL > 10% · FLAG > 5%.
- No stop-loss (when the file has an S/L column): FLAG when over 50% of trades have none.
- Small wins, rare big losses: FLAG when the win rate is over 75% and the average loss is over 3× the average win.
4.4 Not overfitted
- Remove the best 5% of trades: FLAG if the net result is then ≤ 0.
- With 3+ calendar years (each with 10+ trades): FAIL when fewer than half of those years are profitable. With 4+ such years: FLAG when the best 2 years hold over 80% of the net result.
- Too good to be true: FLAG when the Sharpe ratio of daily results (annualised, √252) is over 4, or the win rate is over 90% across 200+ trades. These are the classic signs of look-ahead or a curve fit.
4.5 Not decaying
- Compare the last quarter of trades with the whole record. FAIL when the last quarter's profit factor is under 1.0 while the whole record's is over 1.2 · FLAG when the last quarter's is under 70% of the whole record's.
4.6 Live matches backtest (drift)
Needs both a backtest and a live record of the same strategy; specified with the drift audit (plan section 3b) before it is used.
5. The grade
Computed in code from the check results; the same file always gets the same grade, and payment never changes it.
| Grade | Rule |
| Not enough evidence | Section 3 |
| D — not supported by the evidence | FAIL in Genuine or Survives costs, or 2+ FAILs anywhere |
| C — red flags | 1 FAIL elsewhere, or 3+ FLAGs |
| B — real but with a concern | 1–2 FLAGs |
| A — survives every check | All PASS (N/A allowed for at most one check) |
| B | No FAIL or FLAG, but 2+ checks N/A (too little in the file to clear them) |
Changes
- 2026-11-03: evidence trail and disclaimer v2026-11-03.
- 2026-11-03.1: sections 2–5 pre-registered (readers, minimum evidence, five checks, grade rules) before any real file was graded.
- 2026-11-03.2: clarified before any real file was graded: no FAIL/FLAG but 2+ checks N/A gives B, not A.
- 2026-11-03.3: found on a smoke test of the owner's own statements, before any vendor was graded: MT5 states profit factor after commission and swap (MT4 before), and 2-decimal rounding made equal figures look different. Both now handled as above. Pending stop/limit orders that expired are ignored like cancelled ones.
- 2026-11-03.4: synthetic test, before any vendor was graded: with only 3 years the best 2 hold 67% even when every year is equal, so normal noise tripped the 80% concentration flag on an honest record. The flag now needs 4+ years.