Anthropic Gives Claude 3.5 Sonnet the Ability to Move Your Mouse
Anthropic released Claude 3.5 Sonnet with a computer use feature. The model receives screenshots and issues mouse and keyboard commands through an API loop. Users can automate desktop tasks in apps such as spreadsheets and web browsers without writing code.
This shows that large language models can now act as direct operators of existing software. The change moves automation from script writing to goal setting. Users must shift from writing code to describing outcomes and monitoring safety constraints.
Anthropic documented the feature in its October 2024 announcement. Early testers reported successful navigation through multi step web forms using only natural language instructions.
Step 1: Sign up at console.anthropic.com and enable the computer use beta. Step 2: Send a prompt that describes a desktop task and request the model to control mouse and keyboard. Step 3: Review the action log after each step and adjust the prompt to correct errors.