Benchmarks That Fight Back

In the locally reproduced 20,000-step, 12-seed market-making sweep, a naive two-tick strategy produces P&L of 20,853 with inventory dispersion of 88.1. Inventory skew lowers dispersion to 12.0 but also lowers P&L to 15,448. The Avellaneda–Stoikov calibration at gamma 0.1 loses 15,830.

Those values do not identify a universal winner. They expose an objective trade: gross simulated profit, inventory risk, quote staleness, and toxicity respond differently to each control law.

For a 50-input smooth Black–Scholes workload, pathwise automatic differentiation takes 26.5 milliseconds versus 1,524.5 milliseconds for bump-and-revalue, a 57.6-times local speedup. Delta relative error is 5.56e-16 and the AAD tape costs twice the price calculation.

A sharp digital payoff is the negative control. The pathwise derivative is zero almost everywhere and the resulting Greek is wrong without smoothing or a likelihood-ratio method. The failure is mathematical, not an implementation inconvenience.

TensorForge can prove that a graph rewrite removed nodes, reduced dispatches, and reused buffers because those facts come directly from its IR and allocation plan. It cannot infer a universal wall-time speedup from those structural changes.

KERNELARENA therefore recomputes the structural matrix in the visitor’s browser and runs timing only on that visitor’s CPU and GPU. Adapter, browser, thermal state, and workload size remain part of the result.

Every benchmark record should retain a commit, command, environment, unit, and boundary. Results with different units should not be forced into a common score. Historical records should be appended, not silently rewritten.

A benchmark that can embarrass the preferred technique is doing its job. It tests an engineering claim instead of decorating it.