When the GPU Pays Off: The Parameter-Sweep Roofline, Where a Headline 167x Is Really 27x Algorithm Times 6.2x Hardware

Part of the "Backtests Without Illusions" series.

The speed ladder ended on a deliberately unsatisfying note. We had taken an 80-combo parameter sweep from 69.9 seconds of pandas down to 0.23 seconds of parallel numba on a laptop CPU — a measured 298x — and then argued that a GPU was not the missing rung. The comment section did not accept this quietly, and it was right not to. "Not the missing rung" is a claim about one workload, at one size. It is not a law of nature. So this article does the honest thing and puts the GPU on the bench.

The result is not a verdict, it is a curve. The GPU's lead over the CPU is not a single number you can print on a slide; it is a function of how much work you hand it per call. On our multi-timeframe indicator precompute the GPU's speedup over the CPU engine runs from 54.5x when we ask it for one parameter combination at a time up to 359.6x when we ask for 61 at once. Same kernel, same data, same hardware — the only thing that changed is the batch. A benchmark that reports one of those numbers and hides the other is not measuring the GPU, it is choosing a headline.

And even 359.6x is not what it looks like. Peel it apart and a large multi-timeframe headline of 167x decomposes into 27x of algorithm — a rewrite that also makes the CPU 27x faster — times 6.2x of actual hardware. The GPU did not do the 27x. The math did. This article is about telling those two apart, because conflating them is how a $2,000 graphics card gets sold to solve a problem a git commit would have solved for free.

Provenance: all numbers below are measured on an Apple M2 Max, fp32 on the Metal GPU via MLX versus fp64 numba on twelve CPU threads, from our engine_multitf_gpu.py and bench_param_sweep.py benchmark scripts, each gated by an equivalence check that confirms the GPU and CPU paths produce the same trades. No public companion paper for this one — the scripts are the record.

The question is a roofline, not a benchmark

The reason "how much faster is the GPU" has no single answer is the roofline model (Williams, Waterman & Patterson, 2009). A processor's achievable performance is capped by two ceilings at once: a flat one set by peak compute (FLOP/s), and a sloped one set by memory bandwidth times arithmetic intensity — the number of useful operations you do per byte you move. Cheap-per-byte work lives under the sloped ceiling and is bandwidth-bound; rich-per-byte work reaches the flat ceiling and is compute-bound. The GPU's flat ceiling towers over the CPU's, but its sloped ceiling and, crucially, its fixed cost per call do not scale down as gracefully.

For a parameter sweep, a third term dominates the left side of the chart: launch and transfer overhead. Every GPU call pays a roughly fixed price — dispatch the kernel, move inputs across the (unified, on Apple Silicon, but not free) memory boundary, move results back. Model the wall time for a batch of $B$ combinations as

$T_{\text{gpu}}(B) = O + b\,B, \qquad T_{\text{cpu}}(B) = a\,B$

where $O$ is that fixed overhead, $b$ is the GPU's marginal cost per combo, and $a$ is the CPU's. The speedup is then

$S(B) = \frac{a\,B}{O + b\,B} \;\xrightarrow{B \to \infty}\; \frac{a}{b}.$

This one fraction explains the entire article. At small $B$ the $O$ in the denominator crushes the ratio — you paid to wake the GPU and barely used it. As $B$ grows you amortize $O$ across more combos and the speedup climbs toward its asymptote $a/b$, the true hardware ratio. The half-way point sits at $B = O/b$: a batch-space "ridge point" that tells you how wide your sweep must be before the GPU is even half of what it can be. A sweep of a few dozen combos sits far to the left of that ridge. A sweep of thousands sits on the flat part where the GPU finally earns its ceiling.

So the right question is never "is the GPU faster." It is "is my sweep to the right of the ridge, and is my per-combo work compute-bound enough to reach the flat ceiling when it gets there." Both have to be true. The rest of this article measures where those thresholds actually fall.

The single-timeframe verdict: the GPU barely wins

Start with the workload the speed ladder used: a single-timeframe HMA/HMA3 sweep, 80 combinations over 150,000 bars. We added a sixth rung to that ladder — M5, indicators on the Apple GPU via MLX, trades still extracted on the CPU. Warmed up, best of three, the equivalence gate green:

Against the naive serial baseline the GPU looks heroic — 2,796x. But that is a comparison nobody honest should make: it pits a good GPU implementation against the worst CPU one. Line the GPU up against the CPU you would actually deploy — the same kernel on all twelve cores, M4 — and the win collapses to a sober 3.2x (779 versus 246 combos per second). One entire graphics card, running the whole sweep, beats a twelve-core CPU pool by a factor of three.

Three-point-two is not nothing. It is also not the reason anyone buys a GPU. And it is exactly what the roofline predicts for a sweep this narrow: 80 combos is left of the ridge. The fixed launch-and-transfer overhead $O$ is still a meaningful slice of a 0.10-second job, so we never reach the asymptote $a/b$. Worse, part of the per-combo cost is the O(n) trade-extraction pass, which we deliberately left on the CPU — a term the GPU cannot accelerate at all (more on why in the next section). For a single-timeframe research loop this size, the speed ladder's original verdict stands: the GPU is not the missing rung. Parallel numba already put you at 0.23–0.33 seconds, and shaving that to 0.10 is not what unblocks a researcher. The orchestration around the sweep is.

The interesting word in that verdict is this size. Move right along the batch axis and the story changes.

Where the cost actually lives

Before we scale the batch up, look at what we are actually paying for, because the roofline only rewards you if the expensive part is compute-bound. Profile the sweep and almost all of it is one thing: weighted-moving-average convolutions. An HMA is three WMAs; an HMA3 is four; every combination re-runs them over the whole series. Trade extraction — walk the two indicator arrays, find the sign flips of hma - hma3, book fills — is a single cheap O(n) pass. The sweep is a convolution workload wearing a trading-strategy costume.

That split is exactly the roofline's two regimes:

• Convolutions are compute-bound. Each price is read into many overlapping window sums, so the arithmetic intensity — operations per byte moved — is high. This work reaches for the flat compute ceiling, and the GPU's flat ceiling is the one that towers. Better still, the windows overlap across combinations: a WMA of length 40 is useful to dozens of combos, so a batched precompute shares it once instead of recomputing it per combo. Batching does not just amortize launch overhead; it raises arithmetic intensity by reusing loads. This is the part that belongs on the GPU.
• Trade extraction is bandwidth-bound and branchy. One sequential pass, data-dependent branches at every crossing, essentially no reuse. Its arithmetic intensity is near the floor, and its control flow is hostile to a SIMD device. Pushing it onto the GPU would buy little and cost a lot; it stays on the CPU. Which means it is the serial tail of an Amdahl's-law split — a fixed floor the GPU speedup can never pierce, and part of why the single-TF number saturated at 3.2x.

There is a second, sharper lesson hiding in the multi-timeframe version of this kernel, and it is the source of the 27x we keep promising to explain. The multi-TF engine aligns a higher-timeframe HMA onto the base 1-minute index without look-ahead. Written the obvious way, that is per-bar O(length^1.5) work — recompute the higher-TF moving averages at every base bar. But the aligned HMA is linear in a short buffer of the last few closed higher-TF candles plus the running close, so the entire per-bar computation collapses into a fixed weight vector: one conv1d over the series of closed candles followed by an O(n) gather. Hundreds of millions of redundant operations become a convolution over a far shorter series.

That collapse is an algorithmic win, not a hardware one. It is a better formula. It runs on the GPU, and it runs just as well on the CPU — np.correlate plus a gather, in fp64. Keep that firmly in mind: the single biggest factor in the multi-TF headline is a rewrite available to a machine with no GPU at all. When we finally decompose the 167x, this is the 27x.

The lead grows with batch size

Now the measurement the roofline told us to make. Take the expensive axis — the aligned higher-timeframe HMA precompute on the 1-minute base series, the longest candle stream we have — and feed the GPU an increasing number of length combinations per call, $B = 1, 2, 4, \dots, 61$. The CPU baseline here is the honest production engine: numba with prange across all twelve cores. For each batch we measure both and take the ratio.

This is $S(B) = aB/(O + bB)$ drawn in real measurements. At $B = 1$ the GPU is already 54.5x ahead — because this comparison is against the naive per-bar engine, so the algorithmic collapse is baked in even at one combo — but it is nowhere near its ceiling: the fixed overhead still dominates a one-combo call. Double the batch and the speedup nearly doubles to 102.5x; by $B = 16$ it is 267.4x; by $B = 61$ it is 359.6x and visibly still climbing. The lead grows with the size of the problem. That is the single most important sentence about GPUs and parameter sweeps, and it is the exact opposite of how GPU speedups are usually quoted — as if they were a constant property of the chip.

Two honesty notes, because this is the Backtests Without Illusions series and a clean monotone table always deserves suspicion.

First, the dip: $B = 32$ reads 245.0x, below the 267.4x at $B = 16$. That is not noise to sweep under the rug — it is a chunk-boundary artifact. Our conv1d packs 32 lengths into one kernel chunk, so $B = 32$ fills exactly one chunk with no slack, while $B = 61$ spills into a second chunk that happens to saturate the device better. The point of the roofline is the trend, and a real device has quantization steps in it; we report the wobble rather than cherry-pick around it.

Second, and more important: 54.5x and 359.6x are both against the naive CPU engine, and neither is the hardware win. Both numbers still contain the 27x algorithmic collapse. If you swapped the CPU baseline for the collapsed algorithm running on the CPU — same formula, fp64, both sides optimized — every row would shrink by roughly that factor. Which is precisely the decomposition the next section makes exact.

The honest decomposition: 27x algorithm times 6.2x hardware

To separate the algorithm from the silicon you have to measure three paths on the same multi-timeframe grid, not two. So the benchmark runs:

1. cpu-engine — the production numba engine, per-bar aligned HMA across all cores. The naive-but-parallel baseline.
2. cpu-collapsed — the collapsed weight vectors, np.correlate plus gather, fp64, on the CPU. Same hardware as (1), better algorithm.
3. gpu-mlx — the collapsed weights as a batched conv1d on the Metal GPU, fp32. Same algorithm as (2), different hardware.

Line them up and the full multi-timeframe headline factorizes cleanly:

$\underbrace{167\times}_{\text{gpu vs naive CPU}} \;=\; \underbrace{27\times}_{\text{cpu-collapsed vs cpu-engine}} \;\times\; \underbrace{6.2\times}_{\text{gpu vs cpu-collapsed}}$

The left factor, 27x, is the algorithm — the per-bar-to-convolution collapse from the previous section. It has nothing to do with the GPU. Implement it in numpy and your laptop CPU gets 27x faster on this workload for the price of a refactor. The right factor, 6.2x, is the hardware — the honest, like-for-like win of the Metal GPU over the same optimized algorithm on twelve CPU cores. That 6.2x is the only part you actually needed a GPU to obtain.

This is the whole moral of the article stated as arithmetic. When a vendor benchmark, a library README, or an excited colleague shows you "167x on GPU," the reflex should be a single question: what was the CPU baseline? If the baseline was the naive implementation — and it almost always is, because a slow baseline makes a better slide — then most of the headline is an algorithmic win that the CPU was entitled to as well, and only the residual is hardware. Here the residual is 6.2x. A 167x quote overstates the hardware's contribution by roughly 27-fold.

And notice how the hardware factor itself moved with problem size. On the small single-timeframe sweep the true GPU-versus-best-CPU win was 3.2x. On the larger multi-timeframe precompute it was 6.2x — the same two chips, nearly double the hardware advantage, purely because the bigger workload pushes further up the roofline toward the GPU's flat compute ceiling before the CPU can keep pace. The hardware lead is not a constant either. It is a point on the same rising curve, and the way you move rightward on that curve is to make the batch bigger and the per-combo work richer.

A decision guide: how wide must the sweep be?

Fold the roofline back into a decision you can make before spending money. The GPU pays off when both of the roofline's conditions hold at once: your sweep is right of the batch ridge ($B \gg O/b$, so the fixed launch-and-transfer overhead is amortized), and your per-combo work is compute-bound (rich enough arithmetic intensity to reach the flat ceiling, not a thin O(n) pass). Concretely, from what we measured:

• A few dozen combos of a single-timeframe strategy: skip the GPU. You are left of the ridge; the honest win over parallel numba is ~3.2x on a job that already takes a tenth of a second. The bottleneck is not the kernel, it is everything around it.
• Thousands of combos, or a genuinely multi-timeframe / multi-indicator precompute: the GPU earns its place. The overhead amortizes, the shared convolutions raise arithmetic intensity, and the hardware win climbs to 6.2x and keeps rising with batch. This is the regime where a GPU turns an overnight sweep into a coffee break.
• Climb the CPU ladder first — it is cheaper and it comes first. The 298x on the CPU and the 27x algorithmic collapse are free or nearly so, and they are prerequisites, not alternatives: the GPU's 6.2x is on top of the collapsed algorithm, which you had to write anyway. A GPU bolted onto a naive pipeline mostly measures the naivety.

There is also a tax on the GPU branch that has nothing to do with speed, and you must price it in: Apple's Metal GPU has no fp64 at all. Everything runs in fp32, ~1.2e-7 relative precision. That kills the textbook trick for fast moving averages — the O(n) prefix-sum WMA — because at a price scale near 30,000 over 150k bars the running sums reach ~1e14, seven orders of magnitude past fp32's safe integer range; we measured relative errors up to ~2e2 (a factor of two hundred, not two percent). The working formulation is the direct windowed convolution, where each window sum is a bounded number of comparably sized terms and fp32 stays accurate to ~8e-7. Even then, a strategy that decides on the sign of hma - hma3 will occasionally flip a crossing on a borderline bar where the two curves nearly touch, because fp32 rounding tips a near-tie. That is why the GPU path ships with an equivalence gate that measures how much the trades diverged — PnL delta in basis points, relative shift in trade count — rather than asserting bit-identical output it can never deliver. On our run that divergence was 90 shifted fills out of 479,016 (0.019%), well inside tolerance, but the burden is real: going to the GPU means owning a numerical-parity story, not just a faster clock. That engineering cost is part of the break-even too.

The numbers are Apple-shaped; the curve is not

Every figure above is an Apple M2 Max: a unified-memory device where the GPU and CPU share one pool, and an fp32-only GPU with no double precision. A discrete NVIDIA or AMD card changes the constants, and it is worth being explicit about which way each one moves, because the shape of the argument survives even as the numbers do not.

• Transfer overhead $O$ gets worse, not better. A discrete card sits behind PCIe, so inputs and results make a real copy across the bus that unified memory avoids. That pushes the batch ridge $O/b$ right — you need an even wider sweep before a discrete GPU amortizes its launch. The roofline's left edge is steeper on a PCIe device, not gentler.
• The flat ceiling $a/b$ gets higher. A data-center GPU has far more FLOP/s and bandwidth than an integrated one, so the asymptotic hardware win on a saturating sweep is larger than our 6.2x. The reward for reaching the right side of the curve grows; the toll for sitting on the left grows too.
• fp64 comes back, and with it the prefix-sum trick. On a card with real double precision the O(n) prefix-sum WMA is viable again and the parity gate can be tightened toward bit-exactness. The specific fp32 tax we paid — direct convolution instead of prefix sums, a divergence-measuring gate instead of an assert — is an Apple-Silicon detail, not a law.

None of this changes the thesis. On any device, $S(B) = aB/(O + bB)$: a fixed overhead you must amortize, an asymptote you approach only from the right. The constants are hardware; the curve is arithmetic. Measure your own $O$, $a$, and $b$ with a five-line batch sweep before you trust anyone's headline — including ours.

Where this connects

This is the fourth measurement in a small sub-series about where backtest speed actually comes from, and the pieces fit together as a single argument about what to optimize before you spend:

• The speed ladder climbed from pandas to parallel numba for 298x on the CPU alone, and left the GPU as an open question. This article answers it: the GPU is a real but conditional fifth rung, worth 3.2x–6.2x over the top CPU rung, and only once the sweep is wide enough to climb the roofline.
• The IPC tax made the same move in the other direction — measuring what it costs to leave the process — and reached the same shape of conclusion: the boundary (a socket, a GPU launch) is cheap; the tax is in how often and how chattily you cross it. Batch your GPU calls for the same reason you batch your IPC: to amortize a fixed per-crossing cost.
• The aggregated parquet cache is the CPU-side version of what the GPU precompute does — compute the shared indicators once, reuse them across every combo. The GPU just takes that reuse-and-batch principle to the silicon.
• And the fp32 parity gate is a backtest-live-parity problem in miniature: the moment your fast path computes something slightly different from your reference, you owe a quantified account of the divergence, not a hand-wave.

The connecting discipline is the same one this whole series pushes: measure the thing you are actually being sold. A speedup is a ratio, and a ratio has a numerator and a denominator. Most GPU disappointment comes from a denominator — the CPU baseline — chosen to flatter, and most GPU waste comes from running a sweep too small to leave the left edge of the roofline.

Takeaways

1. A GPU speedup is a curve, not a number. On our multi-timeframe precompute the lead over the CPU ran from 54.5x at one combo per call to 359.6x at sixty-one — same chip, same data. Any single figure is a point on that curve; ask what batch size it was measured at.
2. Always interrogate the CPU baseline. The multi-timeframe headline of 167x factors cleanly into 27x of algorithm (a per-bar-to-convolution collapse, which speeds up the CPU just as much) times 6.2x of actual hardware. The fair GPU-versus-best-CPU win here is 3.2x single-timeframe, 6.2x multi — not 167x.
3. The lead grows with problem size, and so does the hardware factor. Bigger batch and richer per-combo work push you up the roofline: the honest hardware win itself rose from 3.2x to 6.2x purely by making the workload larger. Small sweeps sit left of the ridge and barely benefit.
4. Fix the algorithm and climb the CPU ladder first — the GPU win sits on top of them, not instead. The 6.2x is measured against the collapsed algorithm you had to write anyway. Bolt a GPU onto a naive pipeline and most of what you measure is the naivety, not the silicon.
5. Going to the GPU means owning a numerical-parity story. No fp64 on Metal, the prefix-sum WMA trick dies at price scale (relative error ~2e2), and sign-of-crossing strategies flip on borderline bars. Ship an equivalence gate that quantifies the divergence in basis points; count that engineering cost in your break-even.

When someone tells you the GPU made their backtest a hundred times faster, they have told you almost nothing. Ask them the batch size and the CPU baseline, and the hundred usually resolves into a single-digit hardware win wrapped around an algorithmic one they could have had for free — worth having, on a big enough sweep, for exactly the reasons the roofline says and not one reason more.