Voice

Kollab can listen and speak: an ambient voice bridge turns your speech into chat input and speaks replies back, sentence by sentence, while the terminal interface stays exactly as it is.

Overview

Voice in kollab is a device feature: one process on your machine owns the models, the microphone, the transcript and the speaker. The protocol between that process and your kollab session is local-only and carries no provider credentials.

Ambient speech - speech you direct at the chat becomes input; everything else is ignored
Spoken replies - responses are spoken as finished sentences, with cancellation when you move on
Local transcription - a Whisper base model runs on your device, downloaded automatically on first use
Off by default - nothing listens until you turn voice on

How To Use

Turn voice on in a chat with /voicemode on. Aliases: /vm, /voice.

/voicemode on          # Start listening in this chat
/voicemode off         # Stop voice in this chat
/voicemode status      # Service, microphone and transcript status
/voicemode retry       # Retry setup or pending speech
/voicemode context 20  # Use 20 previous transcript lines (default 10)
/voicemode classifier laya  # Local classifier (or provider)
/voicemodels           # Show automatic voice setup status

The first on runs the automatic setup: it prepares the voice support files, including the speech model downloaded to ~/.kollab/projects/default/voice/. /voicemodels (alias /vmodels) shows where that setup stands.

What Gets Heard

A classifier decides which spoken spans are addressed to the chat. Two classifiers ship: laya, a small local model that runs entirely on the device, and the provider classifier, which uses your configured LLM. Switch between them with /voicemode classifier laya or /voicemode classifier provider; pending speech is preserved across the switch.

Recent transcript lines give the classifier context for each decision, and /voicemode context sets how many (default 10). Ignored speech is kept briefly so a late retry can reconsider it.

Spoken Replies

Replies are queued as sentences and spoken in order; a reply of just . is silent on purpose. Markdown, code blocks and links are stripped before speaking, and moving on cancels what is still queued. If the queue is full a line tells you the reply was not spoken.

Configuration

Voice is off by default. The master switch lives in ~/.kollab/config.json:

{
  "kollabor": {
    "voice": {
      "enabled": false
    }
  }
}

Even when enabled, voice never starts listening on its own after a restart; it waits for /voicemode on in a chat.

Architecture

plugins/voice_plugin.py

The device client and host-side ambient speech bridge. Registers the voice service, RPC methods and its tool; owns admission decisions and transcript context.

packages/kollabor-voice/ (kollabor_voice)

The device service: one process owns the models, microphone, transcript and speaker. Local-only protocol, no provider credentials. Includes the laya classifier worker, the speech queue, audio capture and the automatic setup bootstrap.

handlers/voicemode.py

The /voicemode and /voicemodels commands, under kollabor/commands/system_commands/.