Research2026-08-25
A researcher published a technique and open-source code for starting tool calls early, while an AI agent is still writing the program that will use them. Results are cached so the call returns instantly when the finished code reaches it, and calls that depend on unsafe operations such as reading files are skipped. Measured speed-ups in the author's own tests were modest.
What changed
Harnesses waited for a model to finish writing an entire code block before running any tool calls inside it.
What it unlocks
Starting slow sub-model or search calls while the code that contains them is still being written, and running independent calls in parallel.
- 1-1.2x speed-up on RLM runs
- 8xH100 node, vLLM server
- 5 runs per experiment
What you need to act on it
- a code-execution agent harness
- self-hosted or API model serving
- open-source implementation in Python
Sources