Most Levers Are Wash
Anthropic pointed a model at post-quantum cryptography and it came back with real attacks. The part everybody skipped is the part I have been living in all year.
The writeup is worth reading twice. Claude Mythos Preview found a nontrivial automorphism in HAWK’s lattice, a NIST post-quantum signature candidate, and cut the effective keysize in half. Attack complexity on HAWK-256 dropped from 2^64 to 2^38. Separately it built a fingerprinting technique called the Möbius Bridge that speeds up meet-in-the-middle attacks on 7-round AES by 200 to 800 times.
The process numbers are what stuck with me. HAWK took about 60 hours of a multi-agent harness and roughly $100,000 in API spend. The AES work ran three days and several hundred million tokens, with three substantive prompts from the researchers after the initial nudge.
Then verification took a month.
Discovery got cheap. Trust did not. Everyone quoted the first half of that sentence this week.
I recognized the shape immediately, because I have been running a tiny version of it since spring on matador-miner.
The problem
Matador is a GPU miner for BTX, a chain whose proof of work is a matrix multiply instead of SHA-256d. One static binary, NVIDIA and Apple and AMD, solo or pool. The number that matters is nonce/s, and the number that matters more is nonce/s per watt, because I pay the power bill.
Kernel optimization is a search problem with a merciless verifier attached. There are hundreds of plausible changes. Almost none of them help. And a change that makes the miner faster while making it compute a different answer is not a faster miner. It is a broken one that gets every share rejected.
That is the same shape as cryptanalysis. Huge search space, cheap-ish proposals, expensive truth.
The loop is the upgrade
For the first month I did what everybody does. Ask for an optimization, get an optimization, feel productive, ship nothing measurable.
What actually changed things was writing the loop down.
It lives at .claude/skills/matador-perf/SKILL.md now. Hypothesis, change, build, profile and measure, gate, byte-exact gate, prerelease, release. Eight steps. I wrote it as a skill file specifically because I kept re-deriving it from scratch every single session, badly, and in a different order each time.
That file was a bigger upgrade than any model release. Not a smarter agent. A written-down loop the agent cannot skip steps in.
Grounding beats brainstorming
Anthropic gave Claude Sage and Python and the published literature. That is the whole trick. An ungrounded model produces beautiful nonsense at an incredible rate.
My grounding is the profiler.
There is a document in private/ that opens with a bottleneck model, four facts you re-derive before you are allowed to trust any experiment in the rest of the file. Which stage actually dominates wall time. What the card is thermally and electrically capable of. What the noise floor is. What has already been tried and settled.
Skip that and an agent will confidently optimize a stage that is not the bottleneck, hand you a diff, and explain why it is a win. It will be wrong and it will be articulate. Those two things are unrelated.
The wash rate
Here is the honest ratio. Most levers are wash.
The experiments file exists for exactly one reason, and it says so in the header: to spend the experiment budget on the few that are not. Every entry carries a predicted payoff, an effort cost, a risk, and a gate. Several entries predict “lands within noise” and the stated value is a documented data point saying the current setting is still optimal.
Anthropic burned several hundred million tokens over three days to get one technique. My ratio is not better than theirs. The difference between a productive month and a wasted one was admitting that up front and budgeting for it, instead of being surprised by failure number nine.
The gate is the product
Byte-exact or it did not happen.
A perf win that changes solver output is invalid, full stop, no matter how good the number looks. The gate is a digest probe wired into the release build, plus rej=0 live over a difficulty sweep, plus solved count parity. The build refuses to produce an asset that fails it.
You cannot argue with that gate. Neither can an agent. That is the entire point.
It is the same reason the Jacobian counterexample landed in a week back in July. When the artifact checks itself, you do not have to trust the thing that produced it. If you are going to let agents propose changes to something that costs real money when it is wrong, build the verifier before you build the ideas.
Where agents lie without meaning to
Measurement is the part nobody warns you about.
The 5090 is power capped at 600 watts. A card sitting at 83 to 85 degrees is about 5 percent slower than a cool one. Live pool variance has a noise floor around 2 percent. So an agent hands you a “3 percent win” and what you actually have is a cold GPU and a confident summary.
The rule that fixed it: A-B-A at the same temperature and the same clock, every time. And if you think you found a regression, re-run the old binary in that same state first. If the old binary also moved, it was the GPU, not the code. Sub-2-percent claims do not get to be evaluated on the pool at all, they go to the microbench.
That discipline is my version of Anthropic’s month of verification. It is boring, it is most of the work, and it is the only reason any number in the repo is real.
Same lesson at fleet scale. The public benchmark table is a 19-card sweep, one rented Vast.ai instance per card, July 8 and 9, stock clocks and stock power limit. Every row is a real pool run with zero rejected shares. I rented 19 machines because a benchmark table built from one card and a vibe is worth nothing to the person deciding what to buy.
The best answer was stop
Single-card code is at its tuned floor. That is written into the instructions now, in the imperative, so future sessions do not waste a week rediscovering it. More nonce/s from here is a second GPU or the power knee. It is not more kernel tuning.
An agent that tells you the remaining ideas are wash saves more money than one that ships a 1 percent win.
Two smaller lessons, paid for in real time. Never let an agent build in /tmp, because the box reboots and a day of work is gone, and it will be exactly the day you did not push. And do not launch long jobs over a multiplexed ssh connection, because it wedges the channel and you get silence instead of an error, which is the worst possible failure mode for something running unattended.
The tokens propose. The gate decides. Getting good at this was never about better prompts. It was about building something that says no, and then trusting it more than I trust myself.
— AM