GPT-6 Astra Hits 62.7% On ARC-AGI-3. The Private Adapter Hit 99.9%. Bring Your Own Skepticism.

OpenAI's GPT-6 Astra scored 62.7% on the standard ARC-AGI-3 benchmark. That sounds impressive until you notice their own private adapter reportedly hit 99.9%. The gap between those two numbers is where the honesty lives. Meanwhile, the Kaggle-hosted ARC-AGI-3 agent competition tells a humbler story. Milestone 1, closed June 30, was won by Tufa Labs with an open-source harness called Duck, built on a small Qwen model, scoring just 1.21%. Milestone 2, closed September 30 and announced October 1, went to Daniel Franzen, who adapted Tufa's approach with faster inference, better compute scheduling, and improved agent self-tracking tools.

This is a textbook case of what I call the evaluation gap. Vendors benchmark on their own terms, with their own adapters, and report the flattering number. The independent benchmark tells you what the technology actually does in the wild. The mental model: always distinguish between a private evaluation setup and a public one. The mechanism is contamination through customization. A model with a bespoke adapter is not the same model you will use. The ARC competition results, with scores near 1%, reveal the true frontier of agentic reasoning. It is not 62.7%. It is barely above zero.

OpenAI published the GPT-6 Astra results. ARC Prize runs the benchmark. Tufa Labs won Milestone 1 with their open-source Duck harness on a Qwen model. Daniel Franzen won Milestone 2 by adapting that approach with engineering improvements.