Orchestrator: One API for reliable inference
Connect to open-weight and proprietary AI models through one API with unified billing and automatic same-model failover.

Orchestrator brings open-weight and proprietary models into one API with unified billing. It automatically chooses a provider based on availability, capacity, and latency, and can retry failed requests on another available deployment of the same model.
One integration for open-weight and proprietary models
Orchestrator serves open-weight models hosted on Perplexity's inference infrastructure alongside proprietary models from other providers. The current catalog lists the available model choices.
It accepts OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages requests, so you can connect through your existing OpenAI or Anthropic SDK.
The quickstart and routing guide cover SDK examples and current routing behavior.
Reliability and speed across providers
Reliability
Choosing Claude does not have to mean relying on a single provider: a request that fails on Anthropic can still succeed on AWS Bedrock. Provider failover allows switching the deployment serving a request without changing the model.
Orchestrator uses observed health and capacity to distribute requests among a model's deployments. If a provider returns an error, hits a rate limit or times out, Orchestrator retries on another deployment of that model, with limits on time to first token and total duration for each attempt.
In measurements from October 4 to November 2, 2026, Claude and GPT requests through Perplexity succeeded more often than requests to any individual provider we measured. Their daily success rates also varied less during provider incidents.
We counted a request as failed when Orchestrator exhausted the available deployments or the serving layer returned an error, but excluded client errors such as invalid requests and client rate-limit violations.
Output speed
We’re working on adaptive routing that accounts for model speed so that Orchestrator can adjust provider selection as performance changes.
The fastest provider for any model changes over time. For Claude Opus 5.5, Anthropic led on 15 of the 30 days, and AWS Bedrock led on the other 15.
Over the same 30 days, Azure’s median output speed for GPT-5.6 Sol was 27% higher than OpenAI’s.
One bill across all models
Each request counts only against the answering model’s token usage. Orchestrator charges each model's published per-token rates regardless of which provider serves the request, so provider failover does not change that rate. Usage draws from the same prepaid credits as the rest of your Perplexity API project.
In the API Console, you can view spending by model and filter usage by API key. You can also buy credits through AWS Marketplace.
Automatic routing (preview): match each task to the right model
Orchestrator offers automatic routing, an optional feature available in preview. It selects a model for each request using a profile that pairs plain-language task descriptions with models. You can define your own profile or use Orchestrator’s built-in default.
Automatic routing requests use perplexity/auto as the model. For a custom profile, include a routing object with default_model and routes. Each route specifies a name, description and model.
For each request, perplexity/auto calls our decision model to read the conversation and match it against the route descriptions, without seeing the model names. Orchestrator then sends the request to the model assigned to the selected route, or to your default model if no route matches. Orchestrator also uses your default if the decision model fails or times out.
The examples below cover routing around model refusals, customer-support issue types, and task changes within a conversation.
Routing around refusals
An agent can use automatic routing to avoid predictable refusals while keeping its preferred model for other work. Anthropic documents that Claude Fable 5.2's safeguards can block benign cybersecurity and life sciences work. We tested this use case on a 63-task subset of Terminal-Bench 4.1.
We ran Claude Code with Fable alone and with two custom routing profiles. Both profiles kept Fable as the default and assigned cybersecurity, biology, and after_refusal routes to either GPT-6 Astra or GPT-6.1 Sol. We also included standalone Astra and Sol runs in Codex for comparison.
In this adjusted comparison, both routed Claude Code setups solve 41 of 63 tasks, compared with 38 for Fable alone, at a higher estimated cost per task. The chart shows this trade-off alongside Astra and Sol running in Codex.
The example below uses Sol: the security and biology routes select it before a refusal occurs, after_refusal selects it when the conversation already contains a refusal.
"routing": {
"default_model": "anthropic/claude-fable-5-2",
"routes": [
{"name": "security",
"description": "Work on a cybersecurity or offensive-security task, at any step of it: cracking passwords, hashes or encrypted archives; finding or exploiting vulnerabilities; recovering or extracting secrets, keys or credentials; bypassing security filters or sanitizers; breaking ciphers; malware; or reverse-engineering binaries to recover hidden data.",
"model": "openai/gpt-6.1-sol"},
{"name": "biology",
"description": "Work on a biology or life-science task, at any step of it: DNA, RNA or protein sequences, genome or sequence assembly, primers, plasmids or cloning, molecular or cell biology, pathogens, toxins, or laboratory protocols.",
"model": "openai/gpt-6.1-sol"},
{"name": "after_refusal",
"description": "Earlier in this conversation the assistant refused, declined or could not respond to the task, for example a reply saying the request was refused or blocked.",
"model": "openai/gpt-6.1-sol"}
]
}Routing by issue type
For a customer-support assistant, most questions need only a fast, inexpensive model. A billing dispute calls for a careful, empathetic reply. A broken integration requires troubleshooting. The assistant can use two routes for billing disputes and technical issues, leaving routine questions to its default model:
Route | Task description | Model |
default_model | Everything else, such as how to change an account email | GLM-5.3 Flash |
billing_dispute | A customer disputes a charge, asks for a refund, or is upset about money they were billed. | Claude Opus 5.5 |
technical_issue | A customer reports a bug, an error message, or an integration that stopped working and needs troubleshooting. | GPT-6.1 Sol |
Sending a routed request
The request below sends an API failure issue through the customer-support profile above.
curl https://api.perplexity.ai/orchestrator/v1/chat/completions -i \
-H "Authorization: Bearer $PERPLEXITY_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"prompt_cache_key": "a44efa6b-a43f-4d3a-8922-ed383f40bea7",
"model": "perplexity/auto",
"routing": {
"default_model": "perplexity/glm-5.3-flash",
"routes": [
{"name": "billing_dispute",
"description": "A customer disputes a charge, asks for a refund, or is upset about money they were billed.",
"model": "anthropic/claude-opus-5-5"},
{"name": "technical_issue",
"description": "A customer reports a bug, an error message, or an integration that stopped working and needs troubleshooting.",
"model": "openai/gpt-6.1-sol"}
]
},
"messages": [{"role": "user", "content": "Since this morning every API call returns a 401, even though my key is unchanged."}]
}'The decision model matched the failed API calls to technical_issue, so Orchestrator sent the request to GPT-6.1 Sol. The response includes the selected route and the model that answered:
HTTP/2 200
x-perplexity-autoroute-outcome: routed
x-perplexity-autoroute-route: technical_issue
{"model": "openai/gpt-6.1-sol", "choices": [ ... ], "usage": { ... }}x-perplexity-autoroute-route identifies the selected route, or none. x-perplexity-autoroute-outcome reports how the request was handled, including whether it was routed or fell back to your default model. The model field identifies the model that answered.
Routing within a conversation
As a conversation moves between tasks, the model that handled one turn may not be the right choice for the next. Your follow-up requests include the conversation history, so both the decision model and the selected model can read earlier turns, including replies from other models. We ran three real four-turn conversations: customer support, coding, and research.
Customer support
In the example above, suppose the same conversation moves from an account question to a billing dispute and then a broken integration. As the task changes, the decision model selects the matching route; when the user sends a closing thank-you, Orchestrator returns to GLM-5.3 Flash.
Prompt caching across model handoffs
Each request carries the full message history, but each model maintains its own cache. When the conversation returns to GLM Flash in turn 4, it can reuse a matching prefix from turn 1 while it is still cached, then process the responses added by Opus and Sol in turns 2 and 3. The same goes for Opus or Sol if they are selected in turn 5.
Orchestrator keeps related requests on the same deployment where possible to preserve prompt caching. Reuse a stable prompt_cache_key for the conversation to help maintain that affinity.
Coding assistant
In turn 4 of the coding conversation, GLM-5.3 Flash writes a docstring using the context from Claude Opus 5.5's bug diagnosis and GPT-6.1 Sol's implementation plan.
The routing profile assigns renames and small edits to GLM-5.3 Flash, bug diagnosis to Claude Opus 5.5 and planning changes across modules to GPT-6.1 Sol:
"routing": {
"default_model": "perplexity/glm-5.3-flash",
"routes": [
{"name": "hard_debugging",
"description": "Diagnosing a failing test, crash, deadlock, race condition or performance regression from code, logs or a stack trace.",
"model": "anthropic/claude-opus-5-5"},
{"name": "design_change",
"description": "Planning or reviewing a change that spans several modules, such as an API redesign, a migration or a concurrency model.",
"model": "openai/gpt-6.1-sol"}
]
}Research assistant
In the research example, a factual question leads to a calculation, which then becomes the basis for a memo. The profile assigns factual questions to GLM-5.3 Flash, calculations to GPT-6.1 Sol and memo writing to Claude Opus 5.5:
"routing": {
"default_model": "perplexity/glm-5.3-flash",
"routes": [
{"name": "quantitative_analysis",
"description": "Questions that need calculation, statistics, a financial model or a step-by-step quantitative comparison.",
"model": "openai/gpt-6.1-sol"},
{"name": "long_form_writing",
"description": "Writing a polished memo, report or explanation for a reader, synthesizing earlier findings into a coherent argument.",
"model": "anthropic/claude-opus-5-5"}
]
}The memo in turn 3 uses the calculations from GPT-6.1 Sol's previous reply, rounding the figures without asking the user to repeat them.
Get started with Orchestrator
Connect your existing OpenAI or Anthropic SDK with a Perplexity API key, the appropriate Orchestrator base URL and a model from the catalog.
The quickstart covers setup, current availability and access. For questions, please contact api@perplexity.ai.