Anthropic Gives Claude Direct Control of Your Desktop
Claude 3.5 Sonnet now uses the new Computer Use API to move the mouse, type on the keyboard, and interact with any desktop application. The model receives screenshots and outputs coordinate-based actions to complete tasks such as filling forms or navigating websites. Anthropic reports the feature reaches 14.9 percent success on OSWorld benchmark tasks without additional fine-tuning.
Users stop thinking of AI as a text-only tool and start treating it as an agent that can operate existing software. This shifts workflows from copy-paste prompts toward goal-oriented instructions that the model executes end-to-end.
Zapier is integrating the Computer Use API into its automation platform so non-technical users can trigger desktop actions from natural language commands with measured reductions in manual clicks.
Step 1: Sign up for Anthropic API access and request the computer-use beta at https://www.anthropic.com/api. Step 2: In your code, enable the computer_use_20241022 tool and pass a screenshot plus the task description. Step 3: Run the loop that receives mouse and keyboard actions from Claude and executes them on your local machine until the stated goal is complete.