Anthropic's Claude 3.5 Sonnet Now Manipulates Your Mouse Like an Underpaid Intern
Claude 3.5 Sonnet gained a capability Anthropic calls 'computer use.' The model can perceive your screen, move the cursor, click interface elements, and input text into web forms. This operates through API access, not a local installation, and Anthropic demonstrated it booking flights and filling expense reports.
This teaches the principle of UI-level automation without code. You no longer need to learn scripting languages or API documentation to automate repetitive browser tasks. Your mental model shifts from 'I need a programmer' to 'I need to describe the task clearly to an AI that sees what I see.'
Anthropic, the San Francisco AI lab founded by former OpenAI researchers Dario and Daniela Amodei. They released this as a beta capability alongside Claude 3.5 Sonnet in October 2024.
Step 1: Sign up for Anthropic's API console at console.anthropic.com and request access to the computer use beta. Step 2: Open the provided demo environment, describe a simple task like 'go to a weather website and tell me the forecast for Chicago,' then observe Claude navigating the interface independently. Step 3: Iterate by adding constraints, such as 'find the cheapest flight under $300,' to understand how the model handles ambiguous interfaces.