Blog

The GPU's lead over CPU grows with batch size — 54.5x at one combo per call up to 359.6x at 61 on our multi-timeframe indicator precompute — because a small sweep cannot amortize kernel-launch and transfer overhead. We decompose a headline 167x into a 27x algorithmic win that also helps the CPU and a 6.2x hardware win, show the true GPU-vs-best-CPU lead is only 3.2x single-timeframe and 6.2x multi, and give a decision guide for how wide a sweep must be before a GPU is worth buying into.

Apple's Metal GPU has no float64. Port a vectorized backtest to it naively and the tempting prefix-sum WMA overflows fp32 — max relative error 211× — yet it still runs and returns plausible-looking numbers. The fix is not more precision; it is a different formulation: a direct windowed convolution, fp32-safe to 8×10⁻⁷ and 55.9× faster than single-thread numba. The trap, the arithmetic, and how to prove you didn't fall in.

Drill-down / multi-fidelity search (ASHA, successive halving, Hyperband) screens thousands of configs cheaply and promotes only survivors to the expensive full evaluation. It is a genuine speedup — but it collapses silently if the low-fidelity ranking disagrees with the high-fidelity one. We measured the fold-rank correlation: at one fold Spearman ρ can be 0.03 (ranks almost randomly), climbing to 0.43, 0.67, 0.78, 0.91 as folds accumulate. The fix is one mandatory gate — measure ρ(cheap, full) first, and auto-raise the minimum fidelity to the first rung where ρ ≥ 0.5.

When one backtest is cheap, dumb scrambled Sobol wins on raw throughput — smart samplers (TPE, CMA-ES, ASHA) pay a Python ask/tell tax that drops them 20x, so they evaluate far fewer points at equal wall-clock and lose. Make each eval expensive (multi-TF + walk-forward folds) and the crossover flips. We measured both regimes, and why fold-rank fidelity (ρ@1 rising 0.03→0.43) is the precondition for pruning to pay off.

Not all parameters cost the same to search. A strategy's parameters split into an expensive axis (indicators — recomputed over the whole series) and a cheap axis (decision thresholds — an O(n) pass over precomputed signals). Because indicators are invariant to thresholds, you compute them once and sweep thousands of threshold configs at ~5,600 cfg/s — roughly 1,600x cheaper than recomputing per config. A re-pricing of the curse of dimensionality.

We benchmarked eight backtest engines on one identical parameter sweep — 150k bars, 80 HMA-cross combos, trade-count parity locked at 2707. Two of the most popular event-driven frameworks came in slower than a hand-written pandas loop, while a vectorized/compiled engine ran the same work ~13,000× faster. A study of the per-bar overhead that popular libraries were never built to amortize.

The Deflated Sharpe Ratio prices the winning strategy; PBO prices the search that picked it. Combinatorially Symmetric Cross-Validation runs C(16,8) = 12,870 train/test splits over a 1000x200 performance matrix and asks: does the in-sample winner land in the bottom half out of sample? The catch almost everyone misses — PBO's null is 0.5, not 1. On 200 zero-edge strategies the best in-sample annualized Sharpe of 1.98 collapses to 0.06 out of sample and PBO = 0.476: a coin flip, fully overfit. Plant a real edge (annualized Sharpe 2.38) and PBO drops to 0.001, the in-sample 3.73 surviving to an out-of-sample 2.34. A moving-average grid on a pure random walk has no out-of-sample skill either — PBO 0.463 averaged over 60 matrices, statistically indistinguishable from the null — and on one representative matrix the mirage is vivid: a best in-sample Sharpe of 2.33 collapses to a median out-of-sample -0.22, PBO 0.573, a 63% chance of a loss.