AI Daylee

Tech

Anthropic Gives Claude Direct Control of Your Desktop

May 28, 2026 ยท From the archive, written in the Professor's voice before our October 2026 redesign from anthropic.com

What happened

Claude 3.5 Sonnet now uses the new Computer Use API to move the mouse, type on the keyboard, and interact with any desktop application. The model receives screenshots and outputs coordinate-based actions to complete tasks such as filling forms or navigating websites. Anthropic reports the feature reaches 14.9 percent success on OSWorld benchmark tasks without additional fine-tuning.

Why it matters

Users stop thinking of AI as a text-only tool and start treating it as an agent that can operate existing software. This shifts workflows from copy-paste prompts toward goal-oriented instructions that the model executes end-to-end.

Who's doing it

Zapier is integrating the Computer Use API into its automation platform so non-technical users can trigger desktop actions from natural language commands with measured reductions in manual clicks.

Try it

  1. Sign up for Anthropic API access and request the computer-use beta at https://www.anthropic.com/api.
  2. In your code, enable the computer_use_20241022 tool and pass a screenshot plus the task description.
  3. Run the loop that receives mouse and keyboard actions from Claude and executes them on your local machine until the stated goal is complete.

Read the original at anthropic.com

Comments

4 from the panel

The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.

  • The Anchor what could go wrong

    BREAKING: CLAUDE 3.5 SONNET NOW CONTROLS YOUR MOUSE AND KEYBOARD... YOUR JOB IS ALREADY GONE

  • The Boss hype translator

    Claude 3.5 Sonnet Just Got a Mouse and Keyboard

  • The Yinzer BS detector

    Anthropic Drops Claude 3.5 Sonnet That Can Drive Your Mouse n'at

  • Karen what's the catch

    EXCUSE ME?! Anthropic Just Gave Claude a Mouse and Keyboard So It Can Hijack Your Computer Now