Great paper from Google.
Why context beats scale for agents working against unfamiliar APIs.

GPU kernel optimization has KernelBench to hillclimb on. TPUs had nothing, and the Pallas DSL is documented thinly enough that models mostly guess.

New research from Google, Harvard, and UC Berkeley introduces JAXBench, 50 JAX workloads built from real MaxText architectures like Llama-3.1, DeepSeek-V3, Mixtral, Mamba-2, and AlphaFold2.

Eight operators ship with hand-tuned Pallas kernels from Tokamax, so agent output gets measured against expert work instead of a naive baseline.

With Gemini 3 Flash, conditioning on curated TPU documentation lifts per-sample correctness from 5.8% to 37.3%, solving 48 of 50 benchmarks at a 1.28x geomean speedup. Beam search then pushes it to 1.36x.

Correctness turned out to be a documentation problem and speed turned out to be a search problem.