A microbenchmark can tell us whether an instruction sequence or memory pattern deserves another hour of work. It cannot tell us whether the production worker will execute the same artifact, produce the same result, or improve accepted work over time.
The promotion ladder
- Build identity. Record the source revision, compiler, target architecture, flags, and output hash.
- Semantic equality. Compare deterministic outputs across representative inputs before discussing speed.
- Artifact execution. Prove the worker loaded the candidate binary rather than a cached or fallback path.
- Paired workload timing. Run baseline and candidate in both orders under the same workload and hardware state.
- Live correctness. Observe valid submitted work without treating a successful connection as acceptance.
- Production evidence. Promote only after accepted results and stable operation agree with the local measurements.
build -> equality -> executed artifact -> paired timing -> accepted result -> production
Why the stages stay separate
A local loopback test proves a protocol or result contract. A no-submit live test proves that real jobs can reach the worker. Neither proves that a pool accepted the resulting share. Likewise, an accepted share is a correctness signal, not a statistically useful hashrate comparison.
Keeping those claims separate makes optimization slower at the beginning and much faster at the end. Failed ideas are rejected with a reason, and successful ideas arrive with enough evidence to reproduce them.