Claude 3.5 Sonnet Seizes Your Mouse. It Clicks Where You Cannot. The Autonomy Is Real, The Trust Is Not.
Anthropic's Claude 3.5 Sonnet now operates your mouse, keyboard, and desktop applications directly. The model interprets what is on screen and takes actions across GUI software without requiring you to write a single line of automation code.
This demonstrates what I would call agentic embodiment, where the model does not merely describe a task but executes it through the same interface a human would use. The mental model here is tool use through observation rather than through API calls. The mechanism matters because it means any software with a graphical interface is now programmable by natural language, regardless of whether its developers ever intended that.
Anthropic released Claude 3.5 Sonnet with computer use capabilities, positioning it as a tool for automating everyday desktop tasks.
- Visit anthropic.com and sign up for a Claude account if you do not already have one. Navigate to the chat interface where Claude 3.5 Sonnet is available.
- Describe a simple desktop task in natural language, such as asking Claude to help you organize browser tabs or draft an email in your mail client. Observe how Claude interprets the screen and proposes actions.
- Grant Claude the computer use permission when prompted and watch it attempt to execute your request. Expect imperfect but functional automation that reveals both the capability and the current limitations of GUI-level agent control.