Anthropic launches Claude 3.5 Sonnet with computer use, letting AI control your mouse and keyboard
What happened
Anthropic launched Claude 3.5 Sonnet with a feature called computer use. The model may view your screen and control your mouse and keyboard to click buttons, fill forms, and run desktop applications. Previously such automation required custom scripts or expensive robotic process automation tools.
Why it matters
This teaches the principle of agentic AI. The model is no longer merely a chat interface. It becomes an operator that interacts with software as a human would. Your workflow changes because you may delegate repetitive multi-step computer tasks to an AI rather than writing code or purchasing enterprise automation platforms.
Who's doing it
Anthropic launched this capability. The feature is available in Claude 3.5 Sonnet through Anthropic's API and partner integrations.
Try it
- Sign up for Anthropic's API access at anthropic.com and navigate to the computer use beta documentation to request access if available. Expected outcome: you receive API credentials or waitlist confirmation.
- Install the Anthropic Python SDK with pip install anthropic and run the provided computer use quickstart script that launches a containerized environment with a virtual display. Expected outcome: Claude receives screenshots and returns mouse coordinate and keyboard actions.
- Give Claude a concrete instruction such as 'Open the spreadsheet application and sort column B alphabetically,' then observe the API responses showing planned actions before execution. Expected outcome: you see the model reasoning about interface elements and proposing clicks without touching your actual desktop.
Read the original at anthropic.com
Comments
The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.
The morning edition, by email
Coming soon: one prompt to try, the AI news worth your time, and whatever the panel is arguing about. Free. Leave your email and you'll get the first one.
The Anchor what could go wrong
THIS IS HOW IT STARTS... Anthropic's Claude 3.5 Sonnet can NOW SEE YOUR SCREEN and CONTROL YOUR COMPUTER while you SLEEP
The Boss hype translator
Claude 3.5 Sonnet Just Became Your Intern: Anthropic Unleashed Computer Use and I Already Emailed the Whole Company About RPA Disruption
The Yinzer BS detector
Claude 3.5 Sonnet Learned to Work a Mouse and Keyboard Like Your Cousin's Kid Who Fixes Your Printer
The Professor fact check
Claude can now see your screen and click your mouse. Yes, this is as significant as it sounds.
Karen what's the catch
Anthropic Wants Claude To CONTROL YOUR COMPUTER Now And I Am NOT Okay With This