Skip to main content
Moved to App Settings in v0.129.0

The tuning values on this page are configured in the in-app App Settings screen (sidebar footer, sliders icon) and stored in app_settings.jsonc -- not in .env. A value left in .env is ignored and named once in the startup log; run chatwalaau settings migrate --write to carry an existing configuration across. Feature enable-gates and secrets stay in .env. See Configuration -> Application settings.

Voice & Speech

Voice input (speech-to-text)​

The microphone button records audio and transcribes it through POST /api/transcribe. The transport is auto-selected from the deployment name.

REST path (zero-config)​

For classic Whisper-family models:

WHISPER_DEPLOYMENT_NAME=whisper   # or whisper-1 / gpt-4o-transcribe / gpt-4o-mini-transcribe

The backend calls POST /audio/transcriptions synchronously and returns the transcript -- no extra deployments needed.

Realtime path (gpt-realtime-whisper)​

gpt-realtime-whisper is transcription-only and runs alongside a voice Realtime model, so two deployments are required:

WHISPER_DEPLOYMENT_NAME=gpt-realtime-whisper
WHISPER_REALTIME_CONNECTION_DEPLOYMENT=gpt-realtime-mini   # cheapest voice model

Transport is chosen by the realtime substring in WHISPER_DEPLOYMENT_NAME; override with WHISPER_MODEL_KIND=rest|realtime. Optional knobs:

# Empty selects the GA URL; set a preview value only for legacy models
# such as gpt-4o-realtime-preview.
AZURE_OPENAI_REALTIME_API_VERSION=
# Browser webm/Opus is resampled to this PCM rate. Allowed: 16000, 24000.
WHISPER_REALTIME_AUDIO_RATE=24000

The POST /api/transcribe contract is byte-for-byte identical across both transports, so the SPA is unchanged.

Text-to-speech​

On-demand TTS for any message. The speaker button plays audio and the download button saves an MP3; audio is cached to avoid duplicate calls. Pick the provider with TTS_PROVIDER (default elevenlabs).

Option A -- ElevenLabs:

TTS_PROVIDER=elevenlabs
ELEVENLABS_API_KEY=your-api-key
TTS_MODEL_ID=eleven_multilingual_v2
TTS_VOICE_ID=your-voice-id

Option B -- Azure OpenAI Realtime voice (e.g. gpt-realtime-2), reusing your existing AZURE_OPENAI_ENDPOINT and credentials:

TTS_PROVIDER=azure-realtime
TTS_REALTIME_DEPLOYMENT=gpt-realtime-2
TTS_REALTIME_VOICE=alloy
# TTS_REALTIME_AUDIO_RATE=24000

The Azure Realtime lane reads the message text verbatim (it does not converse), matching ElevenLabs behavior.

Live conversation (GPT-Live)​

New in v0.169.0. Press the Live conversation button (the sound-wave icon next to the microphone) to talk with your agent in real time. Live uses GPT-Live (gpt-live-1 on Azure OpenAI), a full-duplex model: it listens and speaks at the same time, says "mm-hm" while you talk, stops when you talk over it, and waits while you think.

What happens during a Live conversation​

  • Your agent does the work. When you ask for something that needs facts, tools, files or a long answer, GPT-Live hands the task to the agent you selected -- the Built-in agent or one of your Prompt agents -- with that agent's own model, tools, MCP servers and skills.
  • You get two answers. A short spoken summary, and the complete answer, with its tool calls, in the chat. Code and tables are never read aloud.
  • Everything said is written into the chat as messages marked Live. A reply you talked over is marked (interrupted). When Live ends, keep going in the same chat by typing: the agent sees the whole spoken conversation.
  • The message box shows the Live row: your microphone level in teal (Voice Input stays red), the state (Connecting, Live, Working, Muted), the time left, Mute, Send and Stop. Read-aloud (TTS) is paused while Live runs.
  • Mute switches your microphone off on your device and tells the service, so no audio leaves your computer while muted.
  • You can type during Live. What you type joins the same conversation: it appears in the chat as your message and the assistant is told to answer it. Useful for a name, an address or a URL that is hard to say. Attachments and slash commands are not available while Live runs.
  • You can see and stop the work. The row says Working (2) when two tasks are in flight, and each task appears in the chat where its answer will be, with Show progress (collapsed, so a half-written answer does not fill the chat) and Cancel. A cancelled task keeps whatever it had written, and the assistant stops waiting for it.

Where Live is offered​

On the full-page chat (desktop and phone), when the selected run-target is the Built-in agent or a Prompt agent. It is not offered for a Workflow or Harness agent, in the pop-up or side-panel chat, or in demo mode. The microphone needs a secure page (https:// or localhost).

Setting it up​

  1. Deploy gpt-live-1 on an Azure OpenAI resource.

  2. In App Settings > Model Offering Catalog, add an offering in the Live section: provider azure-openai, model ref = your deployment name (for example gpt-live-1). Save. Registering it is what turns Live on -- there is nothing to set in .env and no restart. The offering's endpoint and API-key variable are optional: without them, Live uses AZURE_OPENAI_ENDPOINT and your existing Azure credentials.

    {
      "id": "gpt-live-1",
      "provider": "azure-openai",
      "model_ref": "gpt-live-1",
      "operations": ["live"]
    }

  3. Optional: in Task model assignments, set Live delegation to the chat model the agent should use for tasks Live hands over. Left on Follow session / default, the selected agent's own model is used.

  4. Optional: in App Settings > Speech, pick the Live voice.

At most one Live offering is allowed; it must use the azure-openai provider and cannot serve any other operation.

SettingWhereDefaultMeaning
Live offeringApp Settings > Model Offering Catalog > Livenone (Live off)The GPT-Live deployment. Registering it shows the button.
Live delegationApp Settings > Model Offering Catalog > Task model assignmentsthe selected agent's modelThe chat model delegated tasks run on.
Live voiceApp Settings > Speechmarinalloy, ash, ballad, beacon, bossa, cedar, cinder, coral, delta, echo, gleam, marin, meridian, quartz, ripple, sage, shimmer, stone, tempo, verse, vesper, willow
Live session limit (seconds)App Settings > Speech600A notice appears one minute before the end.
Live silence timeout (seconds)App Settings > Speech60Ends Live after this much silence from both sides.
Live delegation timeout (seconds)App Settings > Speech120Upper bound for one agent task.

Everything applies to the next Live conversation, without a restart.

Privacy and cost​

  • Your Azure credentials never leave the server. The server creates, watches and ends every Live conversation; the browser exchanges only audio with the service.
  • A Live conversation that nobody is watching any more (a closed tab) is ended after 15 seconds, and stopping the server ends them all.
  • Agent tasks started from Live are counted on the Token Usage Dashboard under the lane live, and the time you spent talking is on its Live voice view, per day and per chat. Voice time is kept next to the usage records, so it survives deleting the chat; nothing is priced.