Claude Can Now Move Your Mouse So You Can Fire Your Intern
Anthropic added a computer use API to Claude 3.5 Sonnet that lets the model issue mouse clicks, keystrokes, and screen captures through a sandboxed desktop environment. The model achieved 14.9 percent success on the OSWorld benchmark of real desktop tasks. Developers can now chain these actions into multi-step workflows without writing brittle scripts.
You stop designing automations around brittle API integrations. You start describing the goal and letting the model discover the interface. This changes your workflow from writing glue code to writing goal specifications and reviewing execution traces.
The Ramp finance team built an internal agent that uses Claude 3.5 Sonnet computer use to reconcile expense reports in QuickBooks. They report reducing manual review time from 12 hours per week to under 2 hours with 94 percent accuracy on 500 test transactions.
Step 1: Sign up at console.anthropic.com and enable the computer use beta in your account settings. Step 2: Use the Anthropic Python SDK to create a tool loop that passes screenshots and mouse actions between your local desktop and the model. Step 3: Give the model a task such as open Excel and sum column B then verify the output value appears in your terminal.