Anthropic Gives Claude 3.5 Sonnet Actual Mouse Control

Claude 3.5 Sonnet now controls your screen through a computer use API that moves the cursor, clicks buttons, and types text. The model processes screenshots at 1080p resolution and executes desktop actions without requiring custom scripts or RPA licenses. Users can chain these actions into multi-step workflows for tasks such as data entry or file organization.

This shifts users from writing code to describing outcomes. Workflows that once needed Python scripts or paid automation platforms can now be triggered by natural language instructions. The principle is that large models can replace brittle rule-based automation with flexible, goal-directed interaction.

Anthropic released the feature through their computer use API. Early testers at the AI startup Adept reported completing 40 percent of their internal QA workflows with 70 percent fewer manual clicks.