Anthropic’s Claude 3.5 Sonnet Gains Live Screen Control
Anthropic released computer-use APIs for Claude 3.5 Sonnet that let the model move the mouse, read pixels, and type text on any desktop application. The system records a live screenshot, sends it to the model, and receives coordinate-based actions in return, completing multi-step tasks such as filling forms or running Excel macros without custom scripts.
Readers learn to treat the model as an active operator rather than a text-only assistant. This shifts workflows from writing code to describing outcomes, cutting the barrier between idea and execution for non-programmers.
Freelance operations manager Priya Patel used the beta in production to automate invoice processing for three clients, reducing her weekly admin time from six hours to forty-five minutes.
Step 1: Sign up at https://console.anthropic.com and enable the computer-use preview in account settings. Step 2: Paste the starter Python snippet from the docs and point it at any open window. Step 3: Type a natural-language command such as 'fill the expense report with yesterday’s receipts' and watch the model execute the clicks and keystrokes.