Claude 3.5 Sonnet tops LMSYS with coding and vision gains
What happened
Anthropic released Claude 3.5 Sonnet on 20 June 2024. The model scores 1262 on LMSYS Arena, surpassing GPT-4o by 23 points. It improves coding pass@1 by 18 percent on HumanEval and raises MMMU vision accuracy to 59.4 percent.
Why it matters
Users can now replace multiple specialist tools with one prompt. They stop paying for separate vision APIs. Workflows shift from chaining models to single-model end-to-end tasks.
Who's doing it
Anthropic runs the model on claude.ai and via API. Early adopters at Replit report 30 percent faster code completion rates.
Try it
- Visit claude.ai and select Claude 3.5 Sonnet.
- Upload a screenshot or paste code and ask for a refactor.
- Copy the output directly into your editor and run the tests.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Anchor what could go wrong
BREAKING: ANTHROPIC'S CLAUDE 3.5 SONNET JUST FLATTENED GPT-4O... THIS IS HOW IT STARTS
The Boss hype translator
Anthropic Just Dropped Claude 3.5 Sonnet and It's Beating GPT-4o on Coding and Vision
The Yinzer BS detector
Claude 3.5 Sonnet Tops the Board at LMSYS, Beats GPT-4o at Coding and Vision
Karen what's the catch
EXCUSE ME?! Anthropic Just Dropped Claude 3.5 Sonnet and It's Outperforming GPT-4o on Coding and Vision