OpenAI GPT Live¶
Status: Current | Last reviewed: 2026-10-06
When using ElevenLabs with native voice playback disabled, select Text to Dialogue on the ElevenLabs node and disable sentence aggregation to stream caption fragments as they arrive. Keep your chosen voice and model. This can reduce buffering for longer replies; a short greeting may still be faster with HTTP Streaming, so compare both with your actual opening.
TrueServe preserves audio received while Live connects, so an early caller utterance is not discarded. The greeting starts after that input is delivered; if the caller has already begun speaking, the agent listens instead. If Live accepts the opening but stays silent, TrueServe can replace the connection once after five seconds, provided neither party has spoken and no task has begun. This reduces recovery delay but does not guarantee immediate speech. Start a new Preview conversation after changing voice settings.
GPT Live listens while it speaks and sends work that needs reasoning or tools to a separate backend model. You can choose both models in Agent Studio.
Add GPT Live to an agent¶
- Open a voice agent and add or select its speech-to-speech stage.
- Select OpenAI GPT Live (
GPT Live API) and an OpenAI integration with access to the Live model. - Select GPT Live 1 in Voice model and choose a Voice. Marin is the default.
- Choose Delegation mode on the voice node. A connected backend node appears for both Responses and Client Delegation, with a result connection back to the voice.
- Open the backend node to select its model and write business instructions in Behavior. Use the voice node's Behavior for speaking instructions, greeting and channel capabilities.
- Start a new Preview conversation to test the changes.
Selecting GPT Live does not change your existing Realtime agents.
When adapting an existing agent, bring over its subagents, procedures, variables and capability settings as well as its Behavior prompt. Keep the detailed business steps in Behavior. Use the voice instructions for speaking style and when to ask the backend for help, including answers to a question already asked, corrections, refusals and requests to end the conversation.
For monitored procedures, the backend tracks the current step and completed work. Write each step with its next question and valid stopping conditions, including refusal, no matching record and failed payment. Those outcomes should have an explicit path back to further help or closing; they should not require a successful payment record.
Use Run tests in Preview to select saved conversations and watch their progress. GPT Live tests use synthesized caller speech automatically. Review both the completed actions and what the agent actually said, including its farewell. Use the same expected outcomes when comparing with another model.
The voice dropdown includes all 22 built-in voices: Marin, Cedar, Alloy, Ash, Ballad, Beacon, Bossa, Cinder, Coral, Delta, Echo, Gleam, Meridian, Quartz, Ripple, Sage, Shimmer, Stone, Tempo, Verse, Vesper, and Willow. Where documented, the labels include language, regional influence, and presentation. Bossa and Tempo have Brazilian Portuguese styles; Quartz and Ripple have Australian English styles. Regional influence is a style guide, not an accent guarantee.
Open the delegation branch¶
Every GPT Live node has one connected backend LLM node on its right. The main conversation continues straight down from the voice node to Output. The outgoing connection is Background Thinking; Results returns the answer to the spoken conversation. The backend uses the same card style as other LLM nodes, with a brain icon and an Edit pencil. It has no delete button because its voice node owns the connection. Click either node or its pencil to edit it. These connections are automatic; you do not need to add another audio pipeline branch.
The voice and backend cards have matching dimensions and aligned connection points. Background Thinking and Results run straight across between them in both delegation modes.
The backend card shows the subagent, tool, rule and guard counts for its Behavior configuration. The voice card's Speech Only footer identifies its spoken conversation role and matches the backend footer's appearance.
Responses runs delegated work through OpenAI Live. Its backend node is locked
to OpenAI Responses. In that node's Properties, use Model to
choose the reasoning model independently of the voice. The default remains
gpt-5.6-terra. Only the backend settings supported by Live are shown.
Client Delegation runs the backend on TrueServe's server. Its canvas node and inspector are named Reasoning. Open Reasoning and use the usual LLM selection sequence: Provider, Service, Integration Profile, then Model. It initially uses OpenAI Responses and the voice integration. Choose another profile for separate backend credentials, or change the provider and service. The searchable model list comes from that integration, and a saved model remains selected when you reopen the node. Configure the provider settings below the model. There is no JSON configuration to write and no browser code or callback URL to supply.
Both modes use the same business prompt, enabled tools and capability checks. Changing modes preserves the saved settings for the other mode. Start a new Preview conversation for the change to take effect.
Conversation context in Client Delegation¶
In the backend node's Properties → Conversation context, Previous backend exchanges controls how much history the LLM retains. 0 means full history and is the default. A positive number retains that many previous delegation or application-input exchanges, including their transcripts, answers and complete tool results. The current input and system/capability instructions remain.
This counts backend exchanges, not individual spoken turns. A short window can omit earlier task details; it does not create a summary. Responses delegation manages its own context and does not show this control.
Where each prompt belongs¶
For a conversational step owned entirely by the voice, Client Delegation supports an advanced Silent completion token. Configure the same uppercase token in the backend instructions for that step only. The backend can finish recordkeeping without adding a spoken answer or repeating the voice's question. Keep factual answers, identity clarifications and transfer closings in the backend's normal response. This does not turn off backchannels.
| Editor | Content |
|---|---|
| Voice node → Behavior → Opening greeting | Only the greeting the caller should hear, with variables if needed. |
| Voice node → Behavior → Prompt | Role, speaking style, backchannels, interruptions and when to delegate. |
| Backend node → Behavior → Backend instructions | Business rules, procedures, variables, tool references and automatic capability guidance. |
| Backend node → Behavior → Additional backend instructions | Optional plain-text suffix after the resolved business prompt. Usually leave empty. |
The voice node shows Greeting first, followed by Prompt, using the same editor as other agent prompts. It shows the shared default when no text is saved. Clearing it leaves the editor empty while you write; an empty value uses the shared default when a new session starts.
Backend tools automatically lists enabled tools, backend subagents, and named procedures with their descriptions. The list appears inside the prompt editor and updates when backend configuration changes. It is generated separately from your saved text; changing capabilities does not rewrite your instructions. If your prompt has no Backend tools heading, the list appears at its end.
Each tool entry explains what the operation does and when to use it. For example, collecting a payment, issuing a refund and checking payment status have separate descriptions. Built-in tools use their supplied descriptions; custom tools and MCP operations use the descriptions in their definitions. Long descriptions keep their usage conditions. If a tool has no description, add its purpose and when to use it in the backend tool settings.
Type @ and choose Tool, Subagent, or Procedure to mention existing backend work elsewhere in the voice prompt. These references insert its name and description only. Configure tools and create subagents/procedures in the backend node. A removed reference is described as unavailable; it does not enable a tool. Start a new conversation to use an updated list.
An empty backend prompt offers Add starter structure. Adapt its role, tasks, state handling, result reporting and closing sections to your agent. Existing prompts are preserved. Copy full prompt includes resolved variables, automatic guidance and the optional backend addendum.
Variables, actions and subagents¶
| Feature | Voice node's Prompt | Backend node's Prompt |
|---|---|---|
| Variables and If / Else | Evaluated once at session start, including available lifecycle outputs. | Evaluated and refreshed against session state during the call. |
| Tool, subagent and named procedure references | Names and descriptions only, chosen from existing backend configuration. | Normal backend reference expansion. |
| Run actions, assign variables, define procedures | Configure these in the backend. | Supported through the usual registered tools and checks. |
| Smart subagents | Describe when to delegate in ordinary text. | Add and edit subagents in Behavior. They use the selected backend model and replace its active business instructions. |
This split applies to both Responses and Client Delegation. A smart subagent's Refer mode returns to the prior prompt; Transfer keeps the new role. Switching backend subagents keeps the voice prompt separate. Guided subagents use the existing scripted-flow handoff and return path.
Use backend conditions for facts that change during the call, such as authentication or payment status. Voice variables and If / Else are evaluated at startup. Pasted run/set statements, component references and procedure definitions are identified as backend instructions before Preview.
Tune the conversation¶
Tune reasoning and tool execution in the backend node’s Properties. Both Delegation timeout and Tool timeout are grouped under Timeouts; delegation timeout applies only to Client Delegation.
| Setting | What it controls |
|---|---|
| Reasoning effort and summary | Backend reasoning options supported by the selected model. |
| Backend output token limit | Maximum backend output, including reasoning; leave empty for the model default. |
| Backend service tier | Auto, Default, Flex, Priority, Ultrafast, or Fast tier pilot, subject to model and project support. |
| Backend tool choice | Automatic selection, required tool use, no tools, or one named function. |
| Parallel backend tool calls | Whether the managed backend can request independent tools together. |
| Backend web search | OpenAI-hosted search in Responses mode. |
| Responses text settings | Visual Response verbosity selector: Model default, Low, Medium, or High. |
| Delegation timeout | How long client backend work may run before reporting a timeout. |
| Speak backend progress | Whether client progress updates can be spoken; off by default. |
| Process every caller turn | Send each completed spoken turn to the client backend even if the voice model does not delegate; off by default. |
| Reinforce complete answers | Remind the voice model to preserve every fact, qualification, status and question in final client-backend answers; off by default. |
| Speak tool progress messages | Speak each tool's configured status message before it runs, using the private prompt voice; off by default in both delegation modes. |
| Tool timeout | Default time allowed for a registered tool. |
Leave Speak tool progress messages off to avoid the extra voice interjections between tool calls. You still hear the agent's answers and required payment or secure-capture prompts. To restore the authored tool messages, enable the setting and start a new conversation. Existing GPT Live agents also use the off default when they have no saved value for this setting.
For a Client Delegation agent that must consult its workflow on every caller answer, enable Process every caller turn. The voice model still handles native speech and listening. The application waits for complete captions and runs the existing backend; overlapping native delegations use the same turn once. Private payment, secure-field and guided capture keep ownership of their input. This adds backend work to social turns too, so qualify latency and behavior with the agent's saved audio tests.
For a Client Delegation agent that omits important readback or recap details, try Reinforce complete answers and rerun its saved audio tests. It sends speaking guidance just before each final answer. It does not alter backend actions, speak reasoning or guarantee verbatim delivery. Keep checking what the caller actually hears.
Response verbosity controls how much detail the delegated backend returns. It does not control the voice model's delivery; use the voice node’s Prompt for that. The Live API currently exposes only verbosity in its text settings, so there is no JSON-format or structured-output schema control. If an imported configuration contains unsupported text settings, the editor identifies them and offers Remove unsupported text settings, retaining valid verbosity.
Keep Backend tool choice on Auto for ordinary conversations with capabilities. None prevents tools, including payment and authentication actions. Required or Named function can prevent the backend from giving a plain-text final answer. A timeout does not reverse an action already completed.
Turn Control explains that GPT Live handles speech timing and interruptions. There are no local VAD or interruption tuning controls for this service. Voice isolation, idle prompts, greetings, and background audio remain available.
Put only the greeting you want callers to hear in the greeting field. Names and conditional greetings can use the usual variables. Keep instructions about startup, tools and routing in the agent instructions.
Routine backend instruction refreshes and subagent rule changes do not replay the opening greeting. If an older session repeatedly restarted its greeting, end Preview and start a fresh conversation after the runtime update.
Custom Live connection¶
Leave Voice node → Properties → Advanced connection → Custom Live endpoint empty for normal OpenAI use. TrueServe connects to OpenAI automatically. Only set it when your administrator supplies a Live-compatible proxy endpoint. It is the voice WebSocket connection, not the delegation backend address or a webhook.
Use existing capabilities¶
- Multimodal UI: enable the usual multimodal settings and catalogs. The agent can render A2UI forms and cards and respond to button actions.
- Images and camera: choose a backend with vision support. Uploaded images accompany their text. Camera input supplies the latest snapshot when work is delegated; this is not continuous video understanding by the voice model.
- Payments and secure data capture: configure the existing capabilities and integrations. Private capture uses the existing prompts and protected capture path, then returns the masked result to the conversation. When your Live voice is unavailable in the private TTS service, private prompts use Marin. Set another supported voice under Auxiliary services → Private prompt voice (TTS). This does not change your main Live voice.
- Authentication, tools, knowledge, subagents, and Agent Logic: enable them on the stage as usual. Delegated tools keep their existing permissions, consent checks, argument bindings, and capture restrictions.
- Guided flows: scripted subagents can take over and return control. GPT Live reconnects after the script with its handback context.
- Transcripts and recordings: the existing conversation settings still apply.
Before publishing, test the capabilities your agent actually uses: submit a form, inspect an image, exercise a test payment, switch topics, and complete a guided flow if enabled. Model and integration access are checked against your own provider account when Preview connects.
Configure each capability's own API URL and connection as well as the OpenAI integration. For example, appointment management needs its FHIR endpoint and OAuth integration. A failed lookup can lead the agent to request escalation and end Preview; check the Activity log's tool result and end-session reason.
Cost and changes¶
Cost estimates include the Live session and the delegated backend separately. Backend demand depends on how often the voice model delegates and what tools it uses. Hosted search charges can add to the estimate.
Model, voice, delegation-mode, and configuration changes apply when you start a new conversation. GPT Live always produces speech; it does not expose Realtime's text-only output or VAD settings.
See Taking payments, Secure Data Capture, and Guided flows for capability setup.