TL;DR — In AgentZero Lite, the voice channel is not a chat input. It is the captain's bridge for a fleet of terminal AIs. You speak; an on-device Gemma 4 with a GBNF-locked tool catalog decides which terminal to read, which terminal to write, which key to send, when to wait; and the Akka.NET actor mesh carries those decisions to N concurrent CLI agents — Claude Code, Codex, Cursor CLI, or a plain shell. Same captain, swappable crew. Same bridge, swappable language model behind it. The point of this chapter is to show the seam: how voice becomes a function call, and how a function call becomes a keystroke into another agent's terminal.
This is a follow-up to Adding Providers and a Voice — AgentZero Lite Expands Beyond a Single Local Model. That chapter added STT/TTS as new modalities behind the same gateway. This one shows what the voice channel is for.
You sit at the AgentBot window. The mic toggle is in the input strip. Your workspace already has 2–4 terminal tabs open, each running its own AI CLI (Claude Code in tab 1, Codex in tab 2, a plain PowerShell in tab 3). You say:
"Tell Claude to refactor the auth module and tell Codex to write the tests for it. When Claude finishes, paste its diff into Codex."
The bridge picks it up. Here is what the operator sees:
┌──────────────────────────────────── AgentBot ─────────────────────────────────┐
│ Mode: [AI ▾] Backend: Gemma 4 E4B (local) Reactor: ▶ running R3/12 │
├───────────────────────────────────────────────────────────────────────────────┤
│ │
│ 🎙 YOU — "Tell Claude to refactor the auth module and tell Codex…" │
│ │
│ 🤖 REACTOR — Thinking ░░░░░░░░░ ▓ │
│ ↳ Tool: list_terminals → 3 terminals found │
│ ↳ Tool: send_to_terminal #1 (Claude) "refactor the auth module…" │
│ ↳ Tool: send_to_terminal #2 (Codex) "write tests for auth…" │
│ ↳ Tool: wait 8s │
│ ↳ Tool: read_terminal #1 → captured 142 lines │
│ ↳ Tool: send_to_terminal #2 "<paste diff from Claude>" │
│ ↳ Tool: done │
│ │
│ 🤖 REACTOR — done in 4 turns · 11.2s │
│ │
├───────────── Workspace tabs (each is an independent ConPTY) ──────────────────┤
│ ┌────[1] Claude Code──┬──[2] Codex──┬──[3] PS──┬──[4] +]──────────────────┐ │
│ │ > /refactor auth │ > write tests for… │ PS> _ │ │ │
│ │ ✓ refactored 3 files│ writing test_auth.py │ │ │ │
│ │ diff: 142 lines │ pytest collected 17 │ │ │ │
│ └─────────────────────┴───────────────────────┴──────────┴────────────────┘ │
│ │
│ [🎙 listening · Whisper.net small] ▮▮▮▮▮▯▯▯▯ amplitude · VAD: speaking │
│ │
│ > _ │
└───────────────────────────────────────────────────────────────────────────────┘
Nothing on this screen is a metaphor. The mic strip is a real UserControl (VoiceWaveformIndicator.xaml.cs, 9-bar cyan→magenta gradient driven by VAD amplitude every 50 ms). The reactor log is real ReactorProgress actor messages streamed to the UI thread. The terminal tabs are real ConPTY hosts — each one literally runs claude, codex, or pwsh.exe as a child process. The captain's instructions arrive at those processes as bytes written to a pipe. The terminals don't know they're being driven by an LLM; from their point of view, somebody is just typing.
That last sentence is the load-bearing one. It is why the system works for any CLI AI you can install.
Three roles. Once you see them, the architecture is obvious.
| Role | Component | What it owns | What it doesn't |
|---|---|---|---|
| Captain | The user, via voice | Intent. The desired outcome. | Tactical decisions about which tool to call when. |
| Comms officer | On-device Gemma 4 E4B (or Nemotron 8B fallback) running inside the Reactor actor | Translating intent into a sequence of typed tool calls. | Knowing what each terminal AI does. It only sees terminal IDs and text streams. |
| Crew | N TerminalActors wrapping ConPTY sessions running Claude / Codex / shell | Doing the actual work in their own process. | Each other. They don't share memory or know they have peers. |
The captain speaks in natural language. The comms officer converts that to at most 6 verbs: list_terminals, read_terminal, send_to_terminal, send_key, wait, done. The crew receives plain bytes and produces plain bytes back. That narrow waist is what lets the same UX pilot any future terminal AI without changing the comms officer.
Voice is the riskiest seam in the whole stack: any latency or transcription error here ruins the captain's bridge feeling. The pipeline is built around two principles that fell out of two months of experimentation: dual-VAD gating (so we don't transcribe silence or send half-utterances), and a factory-of-engines behind one interface (so we can swap Whisper.net / OpenAI Whisper / Webnori-Gemma audio / local Gemma audio without code changes downstream).
flowchart LR
Mic[🎙 NAudio<br/>16 kHz mono PCM]
Mic --> RMS[Frame RMS<br/>every ~50 ms]
RMS --> Meter[UI amplitude meter<br/>9-bar gradient]
RMS --> VAD[Utterance VAD<br/>~40 silent frames<br/>= 2 s threshold]
VAD -->|UtteranceStarted| Buf[ring buffer<br/>collect PCM]
VAD -->|UtteranceEnded| Drain[ConsumePcmBuffer]
Drain --> STT{ISpeechToText<br/>factory}
STT -->|local CPU| W[Whisper.net small<br/>~487 MB]
STT -->|cloud REST| O[OpenAI Whisper-1]
STT -->|local server| WG[Webnori-Gemma audio]
STT -->|on-device LLM| LG[Local Gemma audio]
W --> Txt[transcript text]
O --> Txt
WG --> Txt
LG --> Txt
Txt --> Disp[Dispatcher.Invoke<br/>txtInput on UI thread]
Disp --> Send[SendCurrentInput<br/>mode-aware]
Send -->|Chat/Key mode| Term[active terminal write]
Send -->|AI mode| Bot[AgentBotActor.StartReactor]
classDef pri fill:#1e3a5f,stroke:#3794FF,color:#fff
class Bot,Send pri
Implementation pointers: