Anthropic Ships Public Computer-Use API for Claude 3.5 Sonnet
Claude 3.5 Sonnet now exposes an API that moves the mouse, types on the keyboard, and reads the screen in real time. Users send screenshots and goal prompts; the model returns the next mouse or keyboard action until the task finishes. No custom code or browser extensions are required.
This removes the need to write brittle scripts for repetitive desktop work. Teams can now treat any application as an API by describing outcomes instead of clicks. The shift moves effort from code maintenance to prompt design.
Anthropic released the feature on 22 October 2024. Early enterprise testers report completing invoice entry tasks that once took 40 minutes in under 5 minutes.
Step 1: Sign up at https://console.anthropic.com and enable the computer-use beta. Step 2: Send a base64 screenshot plus the text goal to the messages endpoint with tool_use set to computer_20241022. Step 3: Parse the returned action object and execute it on the local machine; repeat until the model signals task complete.