Anthropic Gives Claude 3.5 Sonnet Direct Control of Your Desktop
What happened
Claude 3.5 Sonnet now moves the mouse, clicks UI elements, and types text through a new computer-use API released October 2024. The model receives screenshots as input and outputs mouse coordinates plus keyboard actions. No code or external scripts are required.
Why it matters
Users stop treating the model as a text generator and start treating it as a replaceable worker that performs clicks and keystrokes. Workflow design shifts from prompt crafting to task decomposition and verification loops.
Who's doing it
Anthropic's own research team demonstrated the feature on the public beta at anthropic.com/news/3-5-models-and-computer-use. Early testers report completing invoice data entry in 40 percent less time than manual entry.
Try it
- Visit console.anthropic.com and enable the computer-use preview for your account.
- Send a task such as 'Open Excel and enter last week's sales figures from this CSV' in the API call.
- Review the model's action log and approve or correct each mouse movement before execution.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Yinzer BS detector
Claude 3.5 Sonnet Now Works Your Mouse and Keyboard
Karen what's the catch
EXCUSE ME?! Anthropic Just Gave Claude 3.5 Sonnet the Keys to Click Around on Your Actual Computer Screen
The Anchor what could go wrong
BREAKING: CLAUDE 3.5 SONNET NOW CONTROLS YOUR MOUSE AND SCREEN... THIS IS HOW IT STARTS
The Boss hype translator
Anthropic Drops Claude 3.5 Sonnet Computer Use Feature So We Can Finally Synergize AI Across Our Entire Desktop