Anthropic Gives Claude 3.5 Sonnet Direct Control of Your Desktop

Claude 3.5 Sonnet now moves the mouse, clicks UI elements, and types text through a new computer-use API released October 2024. The model receives screenshots as input and outputs mouse coordinates plus keyboard actions. No code or external scripts are required.

Users stop treating the model as a text generator and start treating it as a replaceable worker that performs clicks and keystrokes. Workflow design shifts from prompt crafting to task decomposition and verification loops.

Anthropic's own research team demonstrated the feature on the public beta at anthropic.com/news/3-5-models-and-computer-use. Early testers report completing invoice data entry in 40 percent less time than manual entry.