Listen to this post (narrated by AI)
Most of the AI agent conversation right now is about software: chatbots, code assistants, browser automation. But a handful of projects are pushing agents into territory where they interact with the physical world, or at least with systems much closer to it than a browser tab. The shift is early and uneven, but it's worth tracking because the practical implications look quite different from anything we've seen with purely software-based agents.
Agents that make phone calls
A project called ClawdTalk (the pun is intentional, it's built around Anthropic's Claude) gives AI agents their own phone numbers. The agent can place and receive actual phone calls, carrying on voice conversations using Claude as the underlying model. If you've ever tried to automate something that ultimately requires calling a vendor, scheduling an appointment, or following up with a human who doesn't use email, you can see why this is interesting.
The architecture combines telephony APIs with real-time speech-to-text and text-to-speech, running Claude in the middle to handle the actual conversation. It's not the first voice AI project, but the focus on outbound phone calls (the agent initiating contact rather than just answering) is a different design point than the typical customer service chatbot.
Whether this becomes practically useful depends on how well it handles the messiness of phone conversations: background noise, interruptions, accents, hold music, automated phone trees on the other end. These are harder problems than text-based chat, and it's too early to say how well they're solved.
Inference on a Raspberry Pi
On the hardware side, a project called PicoClaw took Anthropic's Computer Use capability (which normally requires a Mac Mini or equivalent) and rewrote it in Go to run on significantly less powerful hardware, including devices like a Raspberry Pi.
This matters because it opens up use cases that cloud-based or desktop-based agents can't serve well. Think about a factory floor where you need an intelligent agent at each workstation, a retail environment where devices need to operate offline, or any scenario where sending data to the cloud creates unacceptable latency or privacy concerns. Running inference locally on inexpensive hardware changes the economics and the architecture of these deployments.
The tradeoff is capability. A model running on a Raspberry Pi isn't going to match GPT-4 or Claude running on full cloud infrastructure. But for focused tasks like monitoring a specific process, handling a defined set of interactions, or acting as a smart controller for local equipment, you may not need frontier-model performance. You need something fast, cheap, and locally controlled.
Voice processing moving on-device
There's a broader trend underneath both of these examples: voice AI is moving from cloud-first to local-first. Several open-source frameworks now support on-device transcription and speech synthesis, running the entire pipeline without sending audio data to an external server.
The motivation is partly about latency. Round-tripping audio to the cloud adds noticeable delay that makes real-time conversation feel unnatural. And it's partly about privacy. Voice data is sensitive, and in contexts like healthcare, legal, or personal assistance, keeping that data on the device is a real feature rather than a nice-to-have.
On-device voice models have improved a lot over the past year. The gap with cloud-based systems is narrowing for many practical applications. It's not zero; cloud models still handle edge cases and unusual accents better. But for structured interactions with a limited vocabulary, local processing is often good enough and noticeably faster.
What connects these developments
The common thread across phone-calling agents, edge inference, and on-device voice is that AI agents are expanding their reach beyond text in a browser. Each of these projects is dealing with a different constraint (telephony protocols, hardware limits, latency requirements) and finding that the solutions look quite different from typical chatbot architecture.
For organizations thinking about where AI agents might be useful, this is worth noting. The most visible agent use cases right now are in coding and knowledge work, but a lot of operational work (manufacturing, logistics, field service, customer-facing interactions) happens in environments where a browser-based agent isn't particularly helpful. These projects are early experiments in bridging that gap.
They're early experiments, to be clear. Phone-calling AI still has obvious failure modes, edge devices have real capability limits, and on-device voice processing involves tradeoffs that matter. But the direction is interesting, and the pace of improvement suggests these won't stay experimental for long.