Make the fast loop honest
Aleksey Kladov proposes a practical target for exploratory benchmarks. Adjust the input until one run takes about 300 milliseconds. Milliseconds are easy to scan. The result is slow enough to reduce the influence of one-off startup cost, but fast enough to run several times while changing code.
The point is not laboratory precision. It is a short feedback loop that helps an author choose the next optimization. That distinction matters. A number used to steer one edit needs less ceremony than a number used in a release note, purchasing decision, or regression gate.
Use 300 ms to find the hill. Use a real harness to prove where you landed.
The stopwatch does not define the workload
A longer run cannot repair a bad question. Decide whether you care about cold startup, warm steady state, one request, a batch, wall time, CPU time, memory, or tail latency. Keep the input and host stable while comparing the baseline and candidate.
For command-line work, hyperfine wraps that discipline in a small tool. Its documentation says it runs each command at least ten times by default and chooses a run count aimed at roughly three seconds. It can perform warmup runs, run preparation commands, detect outliers, compare commands, and export JSON. Version 1.21.0 shipped on October 5, 2026. We downloaded its published x86-64 musl archive, ran 30 ms and 50 ms sleep commands with one warmup and three measured runs, then parsed the exported JSON. That smoke check proves the release artifact and command path work here. It does not validate another program or host.
Move from tuning to evidence
Google Benchmark uses a different control model. Its current guide lists a default minimum time of 0.5 seconds for each benchmark, one repetition, and no warmup unless requested. More repetitions add mean, median, standard deviation, and coefficient of variation. Its JSON also records machine context. Criterion.rs stores named baselines so later runs can compare against a fixed reference instead of whichever measurement happened last.
These tools do not contradict the 300 ms rule. They answer a stricter question. The short run keeps a developer in motion. The harness controls timing, repetition, warmup, context, and comparison when the answer must survive review.
Keep the boundary visible
The Hacker News discussion around Kladov's note raised the useful exceptions. JIT warmup, frequency scaling, cache state, long-running leaks, frame drops, and high-percentile latency can sit outside a short sample. Those are not reasons to abandon fast local benchmarks. They are reasons to label them.
Save the command, revision, host, input, warmup policy, and raw output. Record what the run does not cover. If a claimed improvement is small enough to disappear inside the variance, increase the measurement care before increasing the confidence.