#1 ·
Test-Time Compute Scales Better with Parallel Agent Teams
Junlin Wang, Haozhe Chen, Ang Li, Rahul Chalamala
Test-Time ComputeMulti-Agent SystemsInference Scaling
Why it matters
For anyone building agent systems or spending inference compute, this gives a concrete scaling recipe that beats both familiar defaults. The core insight is statistical: many small teams capture the variance gain while keeping a higher mean per sample than independent best-of-N agents, and a single large team gives up that variance benefit entirely. The headline that should change your cost calculus is that, at equal dollar cost, a fleet of small parallel teams of a 27B model beats a frontier model on half of the hard tasks tested.
With a verifier that can score every attempt, the two standard ways to spend test-time compute are best-of-N sampling and one large team of collaborating agents; the first saturates fast and the second stops improving as it grows. Parallel agent teams instead sample many small teams in parallel and keep the best team's result. Across 12 long-horizon tasks (GPU kernels, formal proofs, exploit development), with a 27B model and a 100-agent budget, teams of ten beat parallel agents on 10 of 12 tasks and beat a single team of 100 on all four tasks where that large team was run. A simple statistical model attributes the win to balancing a high mean with the gain from variance.
arXiv | HuggingFace
#2 ·
BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design
Dayou Du, Woon Yee Ng, Guangming Sheng, Nick Lau
KV Cache4-bit QuantizationLLM Serving
Why it matters
If you serve long-context models, FP4 is the format that lets you land on the modern tensor cores, but the naive path is a lose-lose: vanilla quantization drops accuracy, and accuracy-oriented 4-bit codebooks pay a lookup tax in decode throughput. This is the co-design that closes that gap, showing you can run the whole decode-attention path natively on FP4 while staying within about half a point of BF16 accuracy — the kind of result that changes what is actually deployable at scale.
The KV cache dominates memory and bandwidth in long-context serving, and 4-bit microscaling formats (NVFP4, MXFP4) shrink the cache and run attention on FP4 tensor cores, yet existing systems either lose accuracy or lose throughput. BitDecoding-FP4 mitigates three error sources: dual-residual query quantization splits queries into two FP4 tensors, cutting query logit error from 60% to 1%; underflow-aware probability scaling stops E4M3 scales from rounding long-context probabilities to zero; and outlier-aware key quantization combines static smoothing, Hadamard rotation, and scale search. At the system level it removes the softmax bottleneck with transposed softmax plus fused probability quantization and stacked query MMA on paged KV. Across dense and MoE models with GQA and MLA it stays within 0.5 points of BF16 KV accuracy on NVFP4 and tracks BF16 in RL rollouts.
arXiv | HuggingFace
#3 ·
When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents
Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh, Wesley Assunção
Coding AgentsDocumentation DebtAgent Reliability
Why it matters
If you run coding agents on a real repository, your AGENTS.md or CLAUDE.md is now a live part of the system, and this is the first careful measurement of what it costs when that file drifts out of date. The effect is small but compounding — about a third more tool calls on the stale test command — and the most striking detail is that not a single agent repaired the instruction file that misled it. The practical takeaway is to treat these instruction files as versioned code with real maintenance cost, not as static documentation.
Coding agents are steered by repository instruction files read in every session, and like any documentation they go stale; a stale instruction is documentation debt whose interest is the cost added to every later agent session. The authors take three real stale instructions from active open-source repos (a test command, a code location, and a component-adding procedure), have agents fix 22 maintenance tasks under three versions of each — left stale, corrected, or removed — holding the code and tasks fixed, and analyze 732 trajectories at the tool-call level. Agents follow stale guidance when it is relevant, sometimes even when correct information is available elsewhere; on the stale test command Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command. Agents generally recovered from the immediate failures, but none repaired the underlying file.
arXiv | HuggingFace
Outlook
The day's theme is honest accounting for agent systems: where test-time compute actually pays off, where quantized inference still holds its accuracy, and where the scaffolding you wrote down quietly taxes every single run. Expect follow-ups in two directions. On scaling, the mean-versus-variance argument for many small parallel teams is a clean recipe other builders can reproduce at smaller budgets. On reliability, if the file that misleads an agent is never repaired by the agent itself, the maintenance burden lands squarely on the team — treat AGENTS.md as versioned code, not set-and-forget documentation.