Anthropic Ships Public Computer-Use API for Claude 3.5 Sonnet
What happened
Claude 3.5 Sonnet now exposes an API that moves the mouse, types on the keyboard, and reads the screen in real time. Users send screenshots and goal prompts; the model returns the next mouse or keyboard action until the task finishes. No custom code or browser extensions are required.
Why it matters
This removes the need to write brittle scripts for repetitive desktop work. Teams can now treat any application as an API by describing outcomes instead of clicks. The shift moves effort from code maintenance to prompt design.
Who's doing it
Anthropic released the feature on 22 October 2024. Early enterprise testers report completing invoice entry tasks that once took 40 minutes in under 5 minutes.
Try it
- Sign up at https://console.anthropic.com and enable the computer-use beta.
- Send a base64 screenshot plus the text goal to the messages endpoint with tool_use set to computer_20241022.
- Parse the returned action object and execute it on the local machine; repeat until the model signals task complete.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Boss hype translator
Anthropic Just Dropped Claude 3.5 Sonnet Computer Use API and We Need to Synergize This Yesterday
The Yinzer BS detector
Anthropic Gives Claude 3.5 Sonnet a Computer Use API So It Can Click Around Like a Human
Karen what's the catch
EXCUSE ME?! Anthropic just handed Claude the mouse and keyboard and called it progress
The Anchor what could go wrong
BREAKING: CLAUDE 3.5 SONNET NOW CONTROLS YOUR SCREEN. THIS IS HOW IT STARTS.