Claude can now see your screen and click your mouse. Yes, this is as significant as it sounds.
Anthropic launched Claude 3.5 Sonnet with a feature called computer use. The model may view your screen and control your mouse and keyboard to click buttons, fill forms, and run desktop applications. Previously such automation required custom scripts or expensive robotic process automation tools.
This teaches the principle of agentic AI. The model is no longer merely a chat interface. It becomes an operator that interacts with software as a human would. Your workflow changes because you may delegate repetitive multi-step computer tasks to an AI rather than writing code or purchasing enterprise automation platforms.
Anthropic launched this capability. The feature is available in Claude 3.5 Sonnet through Anthropic's API and partner integrations.
Step 1: Sign up for Anthropic's API access at anthropic.com and navigate to the computer use beta documentation to request access if available. Expected outcome: you receive API credentials or waitlist confirmation. Step 2: Install the Anthropic Python SDK with pip install anthropic and run the provided computer use quickstart script that launches a containerized environment with a virtual display. Expected outcome: Claude receives screenshots and returns mouse coordinate and keyboard actions. Step 3: Give Claude a concrete instruction such as 'Open the spreadsheet application and sort column B alphabetically,' then observe the API responses showing planned actions before execution. Expected outcome: you see the model reasoning about interface elements and proposing clicks without touching your actual desktop.