Changed in v0.205.0 -- the speech models moved to the Model Offering Catalog

Voice input and Play message now use a Transcription and a Speech offering in App Settings > Model Offering Catalog, like every other model. The old settings (WHISPER_DEPLOYMENT_NAME, TTS_PROVIDER, TTS_REALTIME_DEPLOYMENT, ...) no longer have any effect, and voice input / Play message answer "not configured" until the offerings exist. Run chatwalaau models migrate-speech to see the offerings your old settings describe, then chatwalaau models migrate-speech --write to add them. Your voice is carried over to the new TTS voice setting automatically.

Voice & Speech

The speech models are Model Offering Catalog offerings: at most one Transcription offering (voice input) and one Speech offering (Play message). Registering one is what turns the feature on, and a change applies to the next request -- no restart. How speech sounds (the TTS voice and the sample rates) is in App Settings > Speech.

Each offering's endpoint and API-key variable are optional: without them it uses AZURE_OPENAI_ENDPOINT and your existing Azure credentials, as the Live and Decision offerings do. A speech deployment in another Azure resource can set its own endpoint and api_key_env.

Voice input (speech-to-text)

The microphone button records audio and transcribes it through POST /api/transcribe. Add a Transcription offering (provider azure-openai). The transport is chosen from the model name.

REST path

For classic Whisper-family models (whisper, whisper-1, gpt-4o-transcribe, gpt-4o-mini-transcribe):

{
  "id": "speech-to-text",
  "provider": "azure-openai",
  "model_ref": "gpt-4o-transcribe",
  "operations": ["transcription"]
}

The backend calls POST /audio/transcriptions and returns the transcript -- no extra deployments needed. api_version defaults to 2024-10-21.

Realtime path (`gpt-realtime-whisper`)

gpt-realtime-whisper is transcription-only and runs alongside a voice Realtime model, so the offering names both deployments:

{
  "id": "speech-to-text",
  "provider": "azure-openai",
  "model_ref": "gpt-realtime-whisper",
  "connection_model_ref": "gpt-realtime-mini",   // the cheapest voice model
  "operations": ["transcription"]
}

A model name containing realtime selects this path; set "transport": "rest" or "realtime" to override. Leave api_version empty for the GA path; set a preview value only for legacy models such as gpt-4o-realtime-preview. The browser's webm/Opus audio is resampled to App Settings > Speech > Realtime STT sample rate (16000 or 24000).

The POST /api/transcribe contract is identical for both transports, so the app is unchanged.

Text-to-speech

On-demand TTS for any message. The speaker button plays audio and the download button saves an MP3; audio is cached to avoid duplicate calls. Add a Speech offering; its provider picks the service.

Option A -- Azure OpenAI Realtime voice (e.g. gpt-realtime-2), reusing your existing AZURE_OPENAI_ENDPOINT and credentials:

{
  "id": "text-to-speech",
  "provider": "azure-openai",
  "model_ref": "gpt-realtime-2",
  "operations": ["speech"]
}

Option B -- ElevenLabs: keep the key in .env (ELEVENLABS_API_KEY=your-api-key) and reference it by name:

{
  "id": "text-to-speech",
  "provider": "elevenlabs",
  "model_ref": "eleven_multilingual_v2",
  "api_key_env": "ELEVENLABS_API_KEY",
  "operations": ["speech"]
}

Set the voice in App Settings > Speech > TTS voice: an Azure voice name such as alloy, marin or cedar (empty means alloy), or your ElevenLabs voice id (required for ElevenLabs). The Azure output sample rate is Realtime TTS sample rate.

The Azure Realtime lane reads the message text verbatim (it does not converse), matching ElevenLabs behavior.

From the command line

chatwalaau models add --id speech-to-text --provider azure-openai \
  --model-ref gpt-realtime-whisper --connection-model-ref gpt-realtime-mini \
  --operation transcription --yes
chatwalaau models add --id text-to-speech --provider elevenlabs \
  --model-ref eleven_multilingual_v2 --operation speech --yes
chatwalaau models migrate-speech           # what the old settings describe (dry run)
chatwalaau models migrate-speech --write   # add those offerings

Live conversation (GPT-Live)

New in v0.169.0. Press the Live conversation button (the sound-wave icon next to the microphone) to talk with your agent in real time. Live uses GPT-Live (gpt-live-1 on Azure OpenAI), a full-duplex model: it listens and speaks at the same time, says "mm-hm" while you talk, stops when you talk over it, and waits while you think.

What happens during a Live conversation

Your agent does the work. When you ask for something that needs facts, tools, files or a long answer, GPT-Live hands the task to the agent you selected -- the Built-in agent or one of your Prompt agents -- with that agent's own model, tools, MCP servers and skills. With Computer Use enabled and Live started on this machine, that work can also operate your Windows desktop (v0.170.0).
You get two answers. A short spoken summary, and the complete answer, with its tool calls, in the chat. Code and tables are never read aloud.
Everything said is written into the chat as messages marked Live. A reply you talked over is marked (interrupted). When Live ends, keep going in the same chat by typing: the agent sees the whole spoken conversation.
The message box shows the Live row: your microphone level in teal (Voice Input stays red), the state (Connecting, Live, Working, Muted), the time left, Mute, Send and Stop. Read-aloud (TTS) is paused while Live runs.
Mute switches your microphone off on your device and tells the service, so no audio leaves your computer while muted.
You can type during Live. What you type joins the same conversation: it appears in the chat as your message and the assistant is told to answer it. Useful for a name, an address or a URL that is hard to say. Attachments and slash commands are not available while Live runs.
You can see and stop the work. The row says Working (2) when two tasks are in flight, and each task appears in the chat where its answer will be, with Show progress (collapsed, so a half-written answer does not fill the chat) and Cancel. A cancelled task keeps whatever it had written, and the assistant stops waiting for it.

Where Live is offered

On the full-page chat (desktop and phone), when the selected run-target is the Built-in agent or a Prompt agent. It is not offered for a Workflow or Harness agent or in demo mode. The microphone needs a secure page (https:// or localhost).

Setting it up

Deploy gpt-live-1 on an Azure OpenAI resource.
In App Settings > Model Offering Catalog, add an offering in the Live section: provider azure-openai, model ref = your deployment name (for example gpt-live-1). Save. Registering it is what turns Live on -- there is nothing to set in .env and no restart. The offering's endpoint and API-key variable are optional: without them, Live uses AZURE_OPENAI_ENDPOINT and your existing Azure credentials.
```
{
  "id": "gpt-live-1",
  "provider": "azure-openai",
  "model_ref": "gpt-live-1",
  "operations": ["live"]
}
```
Optional: in Task model assignments, set Live delegation to the chat model the agent should use for tasks Live hands over. Left on Follow session / default, the selected agent's own model is used.
Optional: in App Settings > Speech, pick the Live voice.

At most one Live offering is allowed; it must use the azure-openai provider and cannot serve any other operation.

Setting	Where	Default	Meaning
Live offering	App Settings > Model Offering Catalog > Live	none (Live off)	The GPT-Live deployment. Registering it shows the button.
Live delegation	App Settings > Model Offering Catalog > Task model assignments	the selected agent's model	The chat model delegated tasks run on.
Live voice	App Settings > Speech	`marin`	alloy, ash, ballad, beacon, bossa, cedar, cinder, coral, delta, echo, gleam, marin, meridian, quartz, ripple, sage, shimmer, stone, tempo, verse, vesper, willow
Live session limit (seconds)	App Settings > Speech	`1200` (20 minutes)	A notice appears one minute before the end.
Live silence timeout (seconds)	App Settings > Speech	`60`	Ends Live after this much silence from both sides.
Live delegation timeout (seconds)	App Settings > Speech	`300` (5 minutes)	Upper bound for one agent task.

Everything applies to the next Live conversation, without a restart.

Privacy and cost

Your Azure credentials never leave the server. The server creates, watches and ends every Live conversation; the browser exchanges only audio with the service.
A Live conversation that nobody is watching any more (a closed tab) is ended after 15 seconds, and stopping the server ends them all.
Agent tasks started from Live are counted on the Token Usage Dashboard under the lane live, and the time you spent talking is on its Live voice view, per day and per chat. Voice time is kept next to the usage records, so it survives deleting the chat; nothing is priced.

Voice input (speech-to-text)​

REST path​

Realtime path (gpt-realtime-whisper)​

Text-to-speech​

From the command line​

Live conversation (GPT-Live)​

What happens during a Live conversation​

Where Live is offered​

Setting it up​

Privacy and cost​