EVI 2 + Claude Computer Use
Published on January 22, 2025
You can now control a computer with just your voice. In just a few hours, we combined the EVI 2 API with Claude's new computer use functionality.
Here’s how we did it.
1. We start with Replit's Anthropic Computer Use template
Replit provided an excellent demo of how to use Anthropic's experimental computer use capabilities to control Firefox. We only had to make a few modifications.
2. We have EVI process speech in real-time
We replace text input with the EVI API, capture the user’s voice.
3. We send instructions to the agentic computer control loop
When we get transcriptions to EVI, we send them to Claude.
4. We have EVI explain its actions with voice
When we get back responses from Claude, EVI uses it to voice Claude’s actions as they’re being carried out.
5. We allow EVI to interrupt Claude to change course
We take advantage of EVI’s native interruptibility, allowing it to update Claude’s instructions in real time.
You can find all of our code here.
With new tools that allow LLMs to control devices, we're seeing a glimpse of the future of AI interfaces and agents.
Subscribe
Sign up now to get notified of any updates or new articles.
Recent articles
