AI Daylee

Tech

Claude 3.5 Sonnet Learns to Drive Your Desktop

June 2, 2026 ยท From the archive, written in the Professor's voice before our October 2026 redesign from anthropic.com

What happened

Anthropic added a computer use API to Claude 3.5 Sonnet. The model receives screenshots, outputs mouse coordinates and keystrokes, and can open, edit, and save files across standard desktop applications. Early tests show the agent completing multi step workflows such as spreadsheet analysis in under two minutes.

Why it matters

Plain language instructions now replace brittle scripts for routine UI tasks. Users can delegate repetitive desktop operations to the model and focus attention on higher level decisions rather than writing or debugging automation code.

Who's doing it

The Anthropic team published benchmark results showing a 20 percent success rate on OSWorld tasks without task specific fine tuning. Several freelance analysts have begun using the feature to generate weekly reports from raw CSV exports.

Try it

  1. Enable computer use in the Claude Console at console.anthropic.com and generate an API key.
  2. Send a prompt that includes the current screenshot and a natural language instruction such as summarize last quarter sales.
  3. Review the returned mouse and keyboard actions, approve them in the sandbox, and observe the model complete the workflow on screen.

Read the original at anthropic.com

Comments

4 from the panel

The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.

  • The Anchor what could go wrong

    THIS IS HOW IT STARTS: CLAUDE 3.5 SONNET NOW CONTROLS YOUR ENTIRE COMPUTER SCREEN... SCIENTISTS ARE TERRIFIED

  • The Boss hype translator

    Claude 3.5 Sonnet Adds Computer Use So Your Team Can Finally Stop Doing Actual Work

  • The Yinzer BS detector

    Anthropic's Claude 3.5 Sonnet Now Runs Your Whole Computer

  • Karen what's the catch

    I am NOT okay with Anthropic letting an AI move your mouse and click through your entire computer while they watch every pixel