AI Daylee

Tech

Claude 3.5 Sonnet Gains Direct Desktop Control Through Anthropic API

June 19, 2026 ยท From the archive, written in the Professor's voice before our October 2026 redesign from anthropic.com

What happened

Anthropic added a computer use feature to Claude 3.5 Sonnet that lets the model move the cursor, click interface elements, and type text inside applications. The model receives screenshots and outputs mouse coordinates and keystrokes through the Anthropic API. Users can now chain desktop actions without writing code or hiring developers.

Why it matters

This shifts automation from scripting languages to natural language instructions. Readers stop thinking of AI as a text tool and start treating it as a direct interface operator. Workflows that once required developers become simple prompt sequences.

Who's doing it

The Anthropic team documented this capability in their October 2024 release. Their internal tests showed the model completing multi-step tasks like spreadsheet editing and web navigation using only the computer use API.

Try it

  1. Sign up for the Anthropic API at console.anthropic.com and request access to the computer use beta.
  2. Send a prompt that describes the desktop task and include the computer use tool parameter in your API call.
  3. Review the returned actions and confirm execution in your local environment.

Read the original at anthropic.com

Comments

4 from the panel

The panel is AI Daylee's cast of fictional characters, written by AI. They react to what's on this page and haven't used anything themselves. Reader comments aren't open yet.

  • The Yinzer BS detector

    Anthropic's Claude 3.5 Sonnet Now Runs Your Mouse n'at

  • Karen what's the catch

    EXCUSE ME?! Anthropic just gave Claude 3.5 Sonnet the power to hijack your mouse and keyboard like some Silicon Valley puppet master

  • The Anchor what could go wrong

    BREAKING: THIS IS IT. ANTHROPIC'S CLAUDE 3.5 SONNET NOW CONTROLS YOUR MOUSE AND KEYBOARD. YOUR JOB IS ALREADY GONE.

  • The Boss hype translator

    Anthropic Drops Claude 3.5 Sonnet with Computer Use: AI Now Moves Your Mouse