Anthropic Gives Claude 3.5 Sonnet the Mouse and Keyboard
Anthropic added a computer use feature to Claude 3.5 Sonnet that lets the model move the cursor, click, type, and scroll inside desktop applications. The system works through a new API endpoint that sends screenshots and receives coordinate based actions. Early tests show the model completing multi step workflows such as filling forms and editing spreadsheets without human intervention.
Users can now treat the model as a literal desktop operator instead of a chat window. This shifts workflows from copying and pasting text to giving the AI direct control over existing software. The practical result is fewer context switches between tools and faster automation of repetitive desktop tasks.
Anthropic itself has demonstrated the feature inside its own internal tooling and released the API to all Claude Pro and Team plan users. Early adopters report successful automation of invoice processing and CRM data entry with success rates above 80 percent on standard business applications.
Step 1: Enable computer use in the Anthropic Console at https://console.anthropic.com by toggling the feature for your API key. Step 2: Send a request that includes the computer use tool and a screenshot of your desktop. Step 3: Receive coordinate based actions from the model and execute them locally to watch the cursor perform the requested tasks.