Anthropic Gives Claude 3.5 Sonnet Direct Mouse and Keyboard Control
Claude 3.5 Sonnet now moves the cursor, clicks buttons, and types text inside live applications. The model executes spreadsheet edits and travel bookings through the same interface a human would use. Anthropic released this capability via their computer use API without requiring custom scripts or additional software.
Users stop writing brittle automation scripts and instead delegate entire desktop workflows to the model. The technique shifts focus from prompt engineering to task specification and verification. Teams can now treat the computer itself as the execution environment rather than an external API.
Anthropic's own research team demonstrated the model completing multi-step booking flows on commercial travel sites. Early enterprise testers report completing spreadsheet reconciliation tasks that previously required dedicated RPA tools.
Step 1: Visit console.anthropic.com and enable the computer use preview in your project settings. Step 2: Send a task description such as 'Open this spreadsheet and calculate quarterly totals' through the computer use API endpoint. Step 3: Review the model's action log and approve or correct each mouse movement and keystroke before execution completes.