Anthropic Gives Claude 3.5 Sonnet a Mouse and Keyboard
What happened
Claude 3.5 Sonnet now uses the computer use API to move the cursor, click, type, and scroll on a real desktop. The model receives screenshots at 1280x800 resolution and outputs structured actions such as left_click[x=450,y=320] or type[text='invoice_2024']. Early tests show it completing multi-step spreadsheet edits and web form submissions without custom scripts.
Why it matters
Users stop treating AI as a text box and start treating it as a desktop agent. The workflow changes from copying model output into software to letting the model operate the software itself, which removes the copy-paste layer and reduces context-switching time.
Who's doing it
Anthropic released the feature in public beta on October 22, 2024. Several small agencies are already routing invoice data entry through Claude, cutting a two-hour manual task to roughly twenty minutes per batch.
Try it
- Sign up at https://console.anthropic.com and enable the computer-use beta in your account settings.
- Run the provided Python SDK example that opens a visible desktop window and passes each screenshot to the model.
- Watch the model complete a three-field web form; the expected outcome is a submitted form with no manual clicks from you.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
Karen what's the catch
EXCUSE ME?! Anthropic just gave Claude permission to hijack your mouse and keyboard like it owns your desktop
The Anchor what could go wrong
BREAKING: CLAUDE 3.5 SONNET NOW CONTROLS YOUR MOUSE AND KEYBOARD... THIS IS HOW IT STARTS
The Boss hype translator
Anthropic Just Dropped Claude 3.5 Sonnet with Computer Use
The Yinzer BS detector
Anthropic Just Gave Claude 3.5 Sonnet Mouse and Keyboard Control