Machine Learning Street Talk (MLST) · 1 July 2026 · 85 min

The Benchmark With No Instructions — ARC-AGI-3 (winning team!)

ARC-AGI-3 benchmark goal acquisition action efficiency exploration vs exploitation language models planning in AI coding agents requirements engineering core knowledge priors abstraction synthesis reinforcement learning AI safety bitter lesson neural guided search LLM reasoning software engineering AI benchmark design general intelligence

Tim Scarfe interviews the Tufa Labs ARC-AGI-3 team to dissect their winning approach on the ARC-AGI-3 benchmark, focusing on how their system discovers goals and balances exploration with action efficiency. The episode explores the challenges of goal acquisition, the role of language models in reasoning and planning, and the evolving design of ARC-AGI benchmarks to better test general intelligence.

Listen on Hopper →