Code mode is batched retrieval

Hornet published Code mode vs tool mode: the agentic retrieval harness matters, and it's the cleanest natural experiment on harness design I've seen written up. They held everything that usually gets the credit constant - same retriever (Hornet), same model (gpt-5.4-mini), same corpus (BrowseComp-Plus, 100,195 documents), same session limits - and changed exactly one thing: how the agent is allowed to issue its retrieval operations.

• Tool mode: one structured operation per model turn. Search. Read. Verify. Curate. Each one is a call, each call is a turn, each turn is a full trip through the context window.
• Code mode: one bounded Python program per turn. Inside that program the model can fire several searches at once, fetch a batch of documents, scan them, and curate what it keeps - all before it spends its next turn.

The result: GoldRecall - the fraction of the known-relevant documents the agent surfaced - went from 0.265 to 0.437, a 65% relative gain. Prompt tokens fell 51%, LLM cost fell 44%. Better and cheaper, from a change that touched neither the model nor the index.

That's the headline, and if you've read the harness is mostly retrieval you can guess how much I enjoyed it. But I want to do the thing I always do, which is take the new words off and see what's underneath.

The new words come off#

"Code mode versus tool mode" is fresh vocabulary. It's riding a wave right now - Anthropic's code execution with MCP, Cloudflare's Code Mode - and the general pitch is about token efficiency: stop shovelling tool schemas and intermediate results through the context window, let the model write a little program instead. That's real, and it explains the token and cost columns cleanly enough.

It does not, on its own, explain the recall column. Writing your orchestration in Python instead of JSON does not make a retriever find more relevant documents. So something else is going on, and the something else is the interesting bit.

Look at what actually moved. Model turns dropped from 33.1 to 12.5. But runtime searches went up, from 28.4 to 44.6. Searches per turn went from 0.86 to 3.57 - a 4.2x increase in how densely the agent probes the index each time it thinks. The win isn't that code mode searched less and got lucky. It searched more, and the harness stopped charging a full model turn for each probe.

So here's the reframe. "Code mode versus tool mode" is new vocabulary for something the field has always had a name for: batched, concurrent probing versus sequential, single-probe retrieval. The independent variable was never Python. It's whether the harness lets the model fan out its retrieval without paying a context round-trip per query.

The harness was rate-limiting recall#

Once you see it as fan-out, the recall gain stops looking like magic and starts looking like classical IR let off the leash.

Trading more probes for more recall is one of the oldest moves in retrieval. Issue query variants and union the results. Turn the ef knob up on your HNSW graph so it explores more of the neighbourhood. Decompose the question and retrieve each part. Every one of these buys recall with additional probing - and every one of them is expensive under a harness that makes each probe cost a model turn.

Tool mode wasn't neutral. It was a rate limiter sitting in front of the retriever, taxing exactly the behaviour that buys recall. Give the model 33 turns and one probe per turn and it rations its searches, because each one competes with reading and reasoning for the same scarce turn budget. Give it a program and it does what any search engineer would do by hand - fires the whole fan of queries at once, because concurrency inside a turn is free. The recall didn't come from somewhere new. It was always available; the harness was suppressing it.

This is the same shape as the recall story in SID-1, from a slightly different direction. There the model was trained to want recall - deliberately tilted to over-report because omission costs a downstream system more than a spurious document does. Here the model was simply permitted to chase recall cheaply. Train the appetite or unblock the plumbing; either way the lever is recall, and it's a retrieval lever.

The lever is general#

The question I find interesting is how far the lesson travels beyond Hornet's setup.

One operation per turn isn't a quirk of their harness - it's the default most agent loops quietly fall into, which is exactly why the result carries. And the lever underneath is fan-out, not code as such. A Python sandbox is one way to issue a spray of queries in a single turn; batched search calls or parallel tool calls are another. The frontier APIs already support parallel tool calls; most agent loops just don't lean on them.

One more thing worth being straight about, and it cuts at my framing rather than Hornet's. Code buys more than fan-out. Hornet names a second mechanism - selective document scanning, inspecting more text without rendering whole documents into the prompt each turn - and there's a third the reframe skates over: adaptivity. A program can search, read what came back, decide what to fetch next and filter it, all inside a single turn. Batched or parallel tool calls get you the parallelism but not that mid-turn loop. So "batched retrieval" is the part of code mode I can name in the old vocabulary, and on this benchmark it looks like the part carrying the recall - but I don't get to pretend it's the whole of what changed.

Which tells you what the lesson actually is. It isn't "write your agent in code." It's let retrieval fan out. Code mode is one way to buy that - a very general one, and if you also want the token savings it's a good bet. But the mechanism doing the heavy lifting on the recall side is batching and concurrency in the retrieval layer, and you can reach for that directly. If your agent probes once per turn, your recall ceiling might not be your embedder or your reranker. It might be your control flow.

The harness is an operating point#

The framing I keep coming back to - from the xAI piece and the answer-engine walk-through - is that every stage of a retrieval pipeline is a choice of operating point on the cost/quality/latency surface. Hornet's result says the harness is one of those stages, and per-turn probe density is one of its knobs. Sequential single-probe and batched fan-out are two operating points on the same curve, and most teams have quietly defaulted to the worse one without ever framing it as a decision.

None of this argues with Hornet. They ran the honest experiment - hold the model, the retriever and the corpus fixed, change only the harness, and report what moved. That is the what, and it stands. The why is the part I've reached for: the missing recall was never locked inside a better embedder or a cleverer reranker. It was sitting behind a harness that charged a full model turn for every probe. Take the tax off and retrieval fans out, the way it always has when you let it. Nothing new arrived here - a single constraint came off, and it moved the numbers further than a new model or a new index would have. Everything is search and search is everything ❤️