OpenAI-Compatible API

ChatWalaʻau exposes the agent as an OpenAI-compatible endpoint so any app using the OpenAI SDK can consume it.

Create an API key with the responses scope (enough for this endpoint) in the API Keys screen, or on the server (Authentication):

chatwalaau keys create --name my-sdk-app --scope responses

Point the SDK's base_url at your instance:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="cwk_xxxxxxxx_...",  # the secret printed once by `keys create`
)

# Non-streaming
response = client.responses.create(
    model="chatwalaau",
    input="What is the weather in Tokyo?",
)

# Streaming
stream = client.responses.create(
    model="chatwalaau",
    input="Explain quantum computing.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)

All agent Tools (Weather, Coding, Image Generation) and Skills are available
Multi-turn conversations via previous_response_id
API sessions appear in the chat sidebar with an API badge
Streaming (SSE) and non-streaming modes

Structured output

The endpoint honors the standard OpenAI text.format field, so you can constrain the answer to JSON or to a specific JSON Schema with your existing SDK code. It is resolved through the same provider seam as the chat UI, so it works for both Azure OpenAI and Anthropic models.

# Constrain to a JSON Schema
response = client.responses.create(
    model="chatwalaau",
    input="Extract the city and the temperature unit from: weather in Tokyo in Celsius.",
    text={
        "format": {
            "type": "json_schema",
            "name": "extraction",
            "schema": {
                "type": "object",
                "properties": {"city": {"type": "string"}, "unit": {"type": "string"}},
                "required": ["city", "unit"],
                "additionalProperties": False,
            },
            "strict": True,
        }
    },
)

# Or just ask for generic JSON
response = client.responses.create(
    model="chatwalaau",
    input="List three primary colors as JSON.",
    text={"format": {"type": "json_object"}},
)

Omit text.format for normal prose output (the default). See Models & Reasoning -> Structured output for the chat-UI side.

Sampling options (`temperature`, `top_p`)

temperature and top_p are accepted on every request and applied per model:

Model routed to	`temperature` / `top_p`
Azure OpenAI, OpenAI (direct), Microsoft Foundry	applied
Claude (Anthropic, direct or Foundry-hosted)	ignored (since v0.192.0)

Since v0.192.0 the Claude provider runs on the Anthropic SDK 1.x, which no longer takes these parameters; the request still succeeds, the values are dropped, and the server log records Ignoring unsupported Anthropic sampling options. Current Claude models with adaptive thinking rejected them anyway. Use the model's reasoning effort instead (Models & Reasoning).

/v1/responses always requires an API key (any scope), regardless of client address, because it is designed for external apps -- see Authentication. Give an external server its own responses-scope key: it reaches this endpoint and nothing else.

Operator controls

Since v0.214.0, three App Settings in the OpenAI-compatible API group bound this endpoint. They apply on the next request and affect /v1/responses only. The defaults keep the earlier behavior.

Setting	Values (default)	Effect
Output cap per model call	0 to 128000 (0 = none)	`max_output_tokens` becomes the smaller of the caller's value and the cap, or the cap when the caller sends none
Tool output in responses	`full` / `sources` / `none` (`full`)	`sources` and `none` remove tool calls and raw tool results from the response; `sources` adds the documents `rag_search` retrieved as annotations
Accepted input roles	`passthrough` / `user_only` (`passthrough`)	`user_only` returns 400 for any `input` item that is not a user message (system, developer, assistant, tool items)

With sources, the answer's text part looks like this:

{
  "type": "output_text",
  "text": "Paid leave is 20 days.",
  "annotations": [
    {"type": "file_citation", "index": 22, "filename": "handbook.pdf", "page": 3, "distance": 0.12}
  ]
}

An annotation means the document was retrieved during the turn, not that the answer used it.
The cap applies to each model call. A turn that uses tools can make several calls, so it is not a cost limit; use the usage limits for that. Reasoning models count their reasoning in it, so a very low cap can return an empty answer.
With user_only, continue a conversation with previous_response_id; the history is kept on the server.
The stored conversation is the same in every mode.

For a company-site chat that answers only from your documents, bind the site's key to a desk agent: see Document desk.

Usage limits

Since v0.216.0, the App Settings group OpenAI-compatible API: limits bounds how much /v1/responses may consume. A rate limit in the calling site (a BFF) protects only the path through that site; these limits also hold when a key leaks. Every limit is 0 (off) by default.

Setting	Effect
`responses_requests_per_minute` / `_per_day` / `_per_month`	Requests admitted per sliding minute, day or month
`responses_input_tokens_per_day` / `_per_month`	Input tokens (cache reads included unless `responses_count_cache_reads` is off)
`responses_output_tokens_per_day` / `_per_month`	Output tokens, reasoning included
`responses_total_tokens_per_day` / `_per_month`	Input plus output
`responses_max_concurrent`	Requests running at the same time
`responses_max_input_chars`	Characters of input per request, stored history included (over it: 400 `input_too_large`)
`responses_limit_timezone`	IANA zone in which days and months reset (default `UTC`, for example `Asia/Tokyo`)

Each key can carry its own limits as well (requests per minute, day and month; total tokens per day and month), set in the API Keys screen (authentication). The stricter limit refuses first.

A refused request runs nothing and gets 429 in the OpenAI error shape, with Retry-After in seconds:

{"error": {"message": "The daily request limit for /v1/responses has been reached.",
           "type": "insufficient_quota", "code": "requests_per_day", "param": null}}

`type`	When	What a BFF can show
`rate_limit_exceeded`	per-minute limit or `responses_max_concurrent`	"Busy, please try again in a moment"
`insufficient_quota`	a day or month limit	"Today's service has ended"

code names the limit (key_ prefix for a per-key limit). The message never names the key.

Keep in mind:

Limits count tokens, not money. With a fixed model, input and output limits track the spend.
Tokens are known only when a turn ends. A request is admitted while any budget remains, so the turns already running can finish above a token limit. Set responses_max_concurrent, responses_max_input_chars and the output cap to bound that.
Counters are kept per server process and rebuilt from the token usage ledger after a restart. With several instances or workers, each counts on its own: divide the limits by the instance count, or run one instance.
A responses-scope key cannot change these settings.
Keep a provider-side quota (for example Azure OpenAI tokens per minute) and a cloud budget alert as the outer guard.

Document desk

Since v0.218.0 a company site's server (a BFF) can run a consultation desk that answers only from the documents you chose, through its own key:

Write a custom agent with a knowledge scope -- the collections and ontologies it may search -- in the agent editor or the YAML (knowledge scope). List only the tools it needs (rag_search, and ontology_query if it uses ontologies) and set grounding: strict and a max_distance.
In the API Keys screen, bind the BFF's key to that agent and set Conversations to Temporary (authentication).
Keep Tool output in responses at sources and the usage limits.

A bound key runs only that agent: only its tools and sources, no web search, never the default model. Send "model": "chatwalaau".

Response	When
400 `model_not_allowed`	a bound key sends any other model name
503 `agent_unavailable`	the bound agent was deleted or has warnings; the key never falls back to another agent
400 `storage_mismatch`	a temporary request (or a temporary key) continues a saved conversation
400 `model_not_found`	an unbound key sends an unknown model while `responses_reject_unknown_model` is on

With strict grounding the server searches the documents before calling the model:

Nothing within max_distance: the reply is a fixed sentence, "I could not find an answer to this question in the documents I can search. Please rephrase your question or contact the person in charge.", no model is called, and the response carries "metadata": {"grounding": "no_match"}.
Otherwise the model answers from the excerpts it was given and the response carries "metadata": {"grounding": "grounded"}; in sources mode the excerpts come back as file_citation annotations. When the model itself replies with the fixed sentence, metadata.grounding is no_match too.

{"id": "resp_...", "output": [...], "metadata": {"grounding": "no_match"}}

On a follow-up turn the search uses the last two user messages, so "and for part-timers?" still finds its subject.

Keep in mind:

Strict grounding guarantees "no documents, no answer". It cannot prove that an answer built from the excerpts adds nothing of its own; that part stays an instruction.
Set max_distance. Without it a collection that holds anything always returns its nearest passages, so no_match happens only for empty collections. Read the distance of the citations to choose a value.
metadata appears only on strict or routed runs; every other response is unchanged.

Conversations:

temporary=true (or a key with Temporary storage) keeps the conversation out of the chat list. Since v0.218.0 a temporary conversation can continue with previous_response_id, so a BFF can use responses_input_roles=user_only and still hold multi-turn conversations. Temporary conversations are deleted after TEMPORARY_CHAT_RETENTION_DAYS.
Saved API conversations can expire: Keep API conversations (days) (responses_session_retention_days, default 0 = forever). Chats from the app are never touched.
previous_response_id is looked up in an index, so the lookup no longer slows down as conversations pile up.

Routing questions to a document area

Since v0.220.0 a desk that covers several business areas can classify each question before it searches. The decision model (the catalog's decision offering, see typed decisions) picks one of the agent's routes, and only that route's collections and ontologies are searched. Fewer near-miss passages from the wrong area reach the model.

Register a decision offering in the Model Offering Catalog, if you have not already.
Give the desk agent two or more routes: in the agent editor under Knowledge > Routing, or in the YAML (routing block). Each route has an id, a description the decision model reads, and a subset of the agent's own sources.
Send "route": "auto" with the request:

client.responses.create(model="chatwalaau", input="How many days of paid leave do I get?",
                        extra_body={"route": "auto"})

What happens:

Decision	What is searched	`route_reason`
a route reaches the minimum probability	only that route's sources	`matched`
no route does	all of the agent's sources, or (with `low_confidence: clarify`) a fixed request for more details and no model call	`low_confidence`
the decision model fails, times out or is not configured	all of the agent's sources, or (with `on_error: fail`) 503 `routing_unavailable`	`error`

The response names the outcome in metadata. All values are strings:

{"metadata": {"grounding": "grounded", "route": "leave", "route_reason": "matched", "route_probability": "0.87"}}

route is a route id, all when every source was searched, or none for the clarification answer. route_probability is absent when the call failed. Log these values in the BFF to find misroutes, then improve the route descriptions or the minimum probability.

Response	When
400 `invalid_route`	`route` is anything but `"auto"`
400 `route_not_available`	`"auto"` from an unbound key, or for an agent without routes
503 `routing_unavailable`	the decision model failed and the agent says `on_error: fail`

Notes:

A request without route behaves exactly as before, with no decision call.
Each routed turn is classified again, with the previous user message included, so a conversation can move from one area to another.
Routing can only narrow the agent's sources, never add to them.
Each routed turn makes one decision call. It waits at most Routing timeout (seconds) (responses_route_timeout_seconds, default 5) and is never retried. Its tokens are recorded under the calling key (purpose responses_route) and count toward the key's token limits; the request still counts once.
The minimum probability defaults to Routing minimum probability (%) (responses_route_min_probability_percent, default 60). An agent can set its own min_probability.

Structured output​

Sampling options (temperature, top_p)​

Operator controls​

Usage limits​

Document desk​

Routing questions to a document area​