Anthropic Gives Claude 3.5 Sonnet the Mouse and Keyboard
What happened
Anthropic added a computer use feature to Claude 3.5 Sonnet that lets the model move the cursor, click, type, and scroll inside desktop applications. The system works through a new API endpoint that sends screenshots and receives coordinate based actions. Early tests show the model completing multi step workflows such as filling forms and editing spreadsheets without human intervention.
Why it matters
Users can now treat the model as a literal desktop operator instead of a chat window. This shifts workflows from copying and pasting text to giving the AI direct control over existing software. The practical result is fewer context switches between tools and faster automation of repetitive desktop tasks.
Who's doing it
Anthropic itself has demonstrated the feature inside its own internal tooling and released the API to all Claude Pro and Team plan users. Early adopters report successful automation of invoice processing and CRM data entry with success rates above 80 percent on standard business applications.
Try it
- Enable computer use in the Anthropic Console at https://console.anthropic.com by toggling the feature for your API key.
- Send a request that includes the computer use tool and a screenshot of your desktop.
- Receive coordinate based actions from the model and execute them locally to watch the cursor perform the requested tasks.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Boss hype translator
Claude 3.5 Sonnet Now Controls Your Screen: The Algorithm Finally Works For Us
The Yinzer BS detector
Anthropic Drops Claude 3.5 Sonnet Computer Use Feature So AI Can Click Around Your Screen Like a Human
Karen what's the catch
EXCUSE ME?! Anthropic just handed Claude 3.5 Sonnet the keys to your entire computer screen
The Anchor what could go wrong
BREAKING: CLAUDE 3.5 SONNET NOW CONTROLS YOUR SCREEN... THIS IS HOW YOUR JOB STARTS VANISHING