Claude 3.5 Sonnet Seizes Your Mouse. It Fills Forms Autonomously. The Autonomy Is Bounded, Naturally.
Anthropic's Claude 3.5 Sonnet now includes a computer use capability where the model controls your mouse and keyboard to perform tasks like moving files, filling forms, or navigating websites. The feature is available now through Anthropic's API and direct release channels. Small businesses can automate repetitive computer work without hiring developers or purchasing enterprise software licenses.
This demonstrates the principle of agentic embodiment, where a language model moves from generating text to taking actions in a graphical environment. The mechanism is tool use extended to human interface elements, meaning the model interprets screen state and emits cursor and keystroke commands. The significance is that automation barriers collapse when the AI uses the same interfaces you do rather than requiring dedicated APIs.
Anthropic released Claude 3.5 Sonnet with computer use capability, detailed at anthropic.com/news/3-5-models-and-computer-use. The model performs mouse and keyboard control for file management, form completion, and website navigation tasks.
- Visit anthropic.com and sign up for an API account or access Claude through the Claude interface. You should see Claude 3.5 Sonnet listed as an available model.
- Describe a specific repetitive task you perform manually, such as organizing files into folders by type or filling out a standard form with provided data. Write your request as a clear sequence of steps with expected outcomes for each.
- Test with a low-stakes task first, such as asking Claude to move three specific files into a named folder on your desktop. Observe how the model interprets your screen and executes actions before attempting anything with sensitive data.