Perplexity Decider

We are releasing Decider v1.3 and Decider Flash v1.3, the successors to our earlier decision models: v1, v1.1, v1.2, and Flash v1.2.

Decider Flash v1.3 is a smaller model distilled from Decider v1.3. Both models are available on Hugging Face under Apache license, through OpenRouter, and the Perplexity Decisions API.

Perplexity Decider is inspired by Jev, TypeSafe AI’s System One Model. It supports binary and multi-class classification with ordered or unordered labels, returning probabilities that applications can use to classify, route, rank, and evaluate inputs.

Both v1.3 models improve performance and robustness across a broad range of text and vision tasks through bidirectional attention, better-curated training data, and a refined training recipe. We also introduce locate, a new feature to highlight the selected content for named entity recognition, personal information identification, and extractive question-answering tasks.

In this post, we explain how we train and evaluate Decider and share how we use it internally at Perplexity.

What is a decision model?

A decision model is a general-purpose classifier. You give it an input and a set of questions each with its possible answers. For each question, it returns a probability distribution over the answers. It supports 3 question types:

• Yes / No: binary decisions.
• Multiple choice: custom multi-class labels.
• Score: ordered levels.

Decision models support a wide range of classification and scoring tasks:

• Judging outputs: assessing faithfulness, quality, and safety.
• Routing: directing requests to smaller or larger models based on difficulty.
• Moderation and security: detecting harmful content, phishing, and prompt injection in tool outputs.
• Filtering and tagging: organizing large collections of content.
• Retrieval and reranking: scoring search results by relevance.

Many of these tasks were previously handled by generative models like LLMs, even though they do not require free-form text generation. Decision models sit between LLMs and task-specific classifiers. They return probabilities for multiple questions about the same input in a single forward pass, avoiding token-by-token answer generation and sharing computation across questions. Unlike task-specific classifiers, they can handle new tasks through natural language prompts, a flexibility learned from training across diverse tasks in a shared question-answering format.

Training

Turning a generative LLM into a decision model

We build Decider by adapting strong generative LLM backbones. To explain the adaptation, we first describe how a request is represented. As illustrated below, a request consists out of a shared context (the state) and a number of questions. The state will be reused by all questions.

For example, given a document and its summary, a single request can ask whether the summary faithfully represents the document and whether it is intended for an expert or a general audience. After the state, each question block contains a question and its possible answers, each assigned a predefined label token such as A, B, or C. A special <|marker|> token indicates where to read out the model’s prediction. At this position, a learned linear output layer (a decision head), maps the hidden state to scores for the question’s answer options, which are then converted to probabilities.

Expand

We initialize the output layer from the base model, which normally scores next-token from the entire vocabulary, and only keep the rows corresponding to the predefined option tokens such as A, B, or C. This pruned layer is then trained to turn the hidden state of <|marker|> into answer scores. This approach covers all classification questions because they can all be formulated as a multiple choice.

For training, we take hundreds of public datasets spanning diverse tasks (sentiment, natural language inference, intent detection, entity matching, moderation, fact checking, ratings, etc.) and convert each example into the unified format described above: a state, one or more questions, their options and a target. The target can be the single correct option if we have gold labels, or a probability distribution over options.

In v1.3, we introduce locate, which extracts text from the state instead of selecting from predefined options. A locate question describes what to find, using either a short label such as “location” or an instruction such as “Extract the city name where the job is located.” The model returns the exact matching text from the state, along with its start and end character locations. If it identifies no matches, it returns an empty list.

Expand

Because of shared state, a single request can combine locate, classification, and scoring questions about the same input. For example, it can ask whether a job requires senior-level experience and extract the required skills.

Bidirectional attention: letting the model read both ways

Our models start from strong pretrained generative backbones, whose causal attention lets each token attend only to itself and preceding tokens. Our ablation studies showed that making the backbone’s full-attention layers bidirectional improves performance, so we adopt this design to let each token incorporate context from the entire input.

Bidirectional attention introduces another design choice: how should the state and questions interact? We considered three attention patterns, with different implications for performance, answer stability, and cache reuse:

• Everything sees everything (v1.2): The state and all questions are processed together with full bidirectional attention. Computation over the state is shared, but an answer to one question can change depending on what else is asked.
• Isolated questions (v1.3): The state attends to all the questions, while each question attends only to the state and its own tokens. Questions can still influence one another indirectly through the shared state but results are more stable.
• Fully cacheable: The state only attends to itself, while each question attends to the state and its own tokens. The state is question-independent and reusable across requests to take advantage of prompt caching. We don't use it by default, because letting the state see the questions matters a lot for several tasks, but a small fine-tuning can switch a model to it.

We ultimately chose isolated questions for their strong performance and stability.

Expand

Training recipe

Training proceeds in three stages for both model sizes. We first train the larger model, based on Qwen3.8-27B, using cross-entropy on large datasets, then distill it into the smaller Qwen3.5-4B model.

The first stage is generalist training, where we fully fine-tune the model on large-scale corpora. In this stage, we only accept data that helps the model across the board and is not specific to any benchmark; we thus exclude any benchmark training splits.

After this generalist model is trained, we run a second, more fine-grained stage using LoRA. The data used in this stage is targeted toward specific domains and tasks. We add data in the second stage rather than the first one if we observe narrow gains or if we observe some regressions on other tasks. We use LoRA to avoid drifting too much from the generalist model and risking losing performance to fit those specific domains. The goal is to improve performance while retaining the backbone’s general knowledge. We also use a weight when merging the LoRA back into the base model, to control the drift more precisely and favor held-out metrics above any in-domain gains.

Finally, we add a vision-specific step. Similar to the second stage, the vision step is treated as another fine-grained specialization step, using a LoRA adapter that is merged back to the base model with a weight. Some vision capabilities can come at the price of text performance. Since we observe much more usage of the models on text, we prioritize text metrics over vision metrics.

For both text stages, we use the average weights from three runs, each using a different random seed. We found this improves the performance and robustness of the aggregated model.

Once the final large model is obtained, we apply the same three-stage training procedure to the smaller model. Here, the large model acts as the teacher: its predicted output distribution replaces the gold labels, and the smaller model is trained to minimize the KL divergence between its predictions and those of the teacher. This distillation objective outperforms cross-entropy on the same training data. While cross-entropy performs better on benchmarks with dedicated training data, it generalizes less well to held-out data. We hypothesize that distillation better transfers the teacher’s output distribution, including its confidence over non-gold alternatives, whereas cross-entropy encourages overfitting to the benchmark-specific training distribution. We therefore adopt the KL-based distillation objective.

To mitigate train-test contamination, we remove any training samples that could overlap with any evaluation we use. This is important because some public training datasets contain large parts of some evaluation datasets. We also conduct extensive bias audits and removal to make sure our gains are reliable and not just driven by learned biases.

Adding the ability to extract information

The locate feature extracts information such as named entities from text, without changing how it answers ordinary requests. To do so, we introduce an additional training stage: we add a new special token, <|locate|>, a span head, and LoRA adapter to Decider.

We initialize <|locate|> ’s embedding from <|marker|> learned in the previous training stages. We train only this new embedding, the span head, and the LoRA adapters, while keeping the original backbone weights and decision head frozen. Each requested field gets its own <|locate|> token.

The span head uses <|locate|> ’s representation together with the state-token representations to score where matching spans start and end, and which start-end pairs belong together. It also uses the <|locate|> token’s representation to predict field-specific thresholds for these scores. Selected spans are returned as character positions in the original text.

We train the span head with an adapted InfoNCE loss that pushes every gold span above the threshold, and the threshold above every other span. The loss has two terms: The first one pushes the scores of correct spans above the threshold and the wrong spans. The second term pushes the threshold above the scores of wrong spans. Fields with no answer get only the second term, which teaches the model to return an empty list.

We train on a mix of span extraction tasks, including named entity recognition, PII detection and extractive question answering. We vary the field descriptions from short labels to detailed instructions, so the model learns to follow the request rather than memorize a fixed label set.

Locate teaches the model a structurally different task, and we want to add it without losing what the model already does well. Merging the locate adapters into the backbone causes some regression on decision requests. To limit this, we are running investigations such as more replay data from earlier stages and auxiliary objectives that keep the model’s outputs on decision requests close to those of the original model. However, we find that merging is currently still not lossless, so we keep the adapters unmerged and activate them only for locate requests.

Evaluation

We evaluate Decider v1.3 and Decider Flash v1.3 on external benchmarks from the Hugging Face Decision Index, Vals AI’s ClaimBench, and Artificial Analysis Decision Index.

Across these evaluations, our models demonstrate strong performance. Even though these leaderboards include privately held tasks, they can still be gamed through repeated submission and iterative tuning based on leaderboard feedback. To complement these leaderboards, we evaluate the models on an internal suite combining private production tasks with held-out evaluations drawn from public non-benchmark datasets. These evaluations provide an additional check on generalization beyond public benchmarks, helping us distinguish broader improvements from gains specific to familiar evaluation suites.

Internally, we built a suite of more than 200 tasks spanning various domains and tasks such as classification, scoring, reranking, and entity extraction. To honestly assess Decider’s real-world performance, all evaluations are performed zero-shot, without task-specific fine-tuning, giving us a clear signal of where to improve the model.

Query classification

At Perplexity, we use a multi-head classification system to process each user query. Each query is first passed through an embedding model that acts as a feature extractor. The resulting embedding is then fed into multiple separate classification heads, including both binary and multi-label classifiers. For example, one head determines whether a user query should trigger web search.

Across 63 classification tasks with independently relabeled references by a frontier LLM, Decider v1.3 achieves 91.6% mean accuracy, exceeding our production classifiers by 6.54 percentage points. Decider Flash v1.3 also outperforms the production baseline, while Decider v1.3 outperforms Jev 1.14.

Expand

Domain-specific classification

We assembled a domain-specific evaluation suite from public datasets to assess decision models’ classification performance across seven domains: claim verification (6 tasks), healthcare (7), cybersecurity (8), programming (16), government (7), law (9) and finance (9).

For example, “judge the time complexity of a Python program”, or “use an annual financial report to determine whether a company’s unrestricted cash covers debt principal due within the next twelve months”. Decider v1.3 leads across all seven domains, outperforming Jev 1.14 by up to 12.9 percentage points in finance and 9.3 points in law. Decider Flash v1.3 trails Jev 1.14 in finance, programming, and law, and performs similarly in other domains.

Expand

Retrieval and Reranking

Search is at the core of Perplexity. We maintain a broad suite of retrieval tasks to evaluate our models’ reranking performance. For each task, we first use a pool of embedding models to retrieve the top 50 candidate documents from each corpus. We then rerank these candidates using Decider and compare its performance against a strong reranker baseline, Qwen3-Reranker-8B.

We evaluate Decider in a zero-shot, pointwise setting, using the model to directly score each query–document pair independently and rank documents by relevance. Across 72 retrieval tasks, Decider v1.3 achieves a mean nDCG@10 of 83.99, outperforming Qwen3-Reranker-8B by 1.81 points and Jev 1.14 by 2.24 points.

On the BEIR benchmark, both models outperform Qwen3-Reranker-8B by 8.78 and 7.05 nDCG@10 respectively. Against Jev 1.14, Decider v1.3 is 1.33 points higher, while the Flash version scores 0.4 lower.

Expand

Multilingual classification

To evaluate decision models beyond English, we assembled a multilingual classification benchmark comprising 19 language-specific tasks from 8 public datasets, with 46,484 examples across eight languages: German, Spanish, French, Italian, Russian, Japanese, Korean, and Chinese. The benchmark covers intent detection, topic classification, sentiment analysis, natural language inference, paraphrase detection, offensive language detection, and grammatical acceptability.

We report mean accuracy across tasks within each language, weighting each task equally. Decider v1.3 outperforms Jev 1.14 in all eight languages and Decider Flash v1.3 does so in six, with mean task accuracies of 88.16% for Decider v1.3 and 86.50% for Decider Flash v1.3, compared with 85.41% for Jev 1.14.

Expand

NER, PII and Extractive QA

We evaluate the locate feature on 17 out-of-domain tasks that test its extraction across various domains, input formats (text, tables, structured documents) and instructions (short labels, questions and detailed descriptions). The domains range from Wikipedia to scientific papers, cyber-threat reports, technical support, privacy policies, resumes, and legal contracts. The requests range from short label names such as “person” or “email address”, through questions about a table or a support ticket, to detailed field descriptions for structured extraction, such as the parties of an NDA or the total on an invoice.

We evaluate NER and PII using exact-span F1, which requires predicted spans to match the reference boundaries exactly. Extractive QA uses character-level F1, giving partial credit for overlapping answer text, while structured extraction uses field accuracy, the proportion of fields extracted correctly. For CV parsing, we average the NER and PII exact-span F1 scores.

We compare our models against general-purpose extraction models of different sizes: GLiNER large v2.5 (0.5B), Liquid d1 (2B) and Cloudflare’s Clairic (undisclosed). Decider v1.3 reaches an average score of 69.0 (using each task’s primary metric) and Decider Flash v1.3 reaches 63.6, compared with 43.3 for Liquid d1, 41.5 for Clairic and 33.6 for GLiNER large v2.5. Both Decider models lead in every category.

Expand

Image classification

Decider has natively supported vision tasks since v1. To evaluate its performance beyond text, we assembled a five-task benchmark suite with a particular focus on document classification, including invoices, letters, and resumes. The suite also covers satellite image classification, texture classification, pet breed classification, and animal counting.

Decider v1.3 achieves a mean accuracy of 76.99% across the five visual tasks, outperforming Jev 1.14 by 1.47 percentage points. Decider Flash v1.3 achieves 72.77%, demonstrating visual classification capabilities in a smaller model.

Expand

Real-world decision requests

We sampled one million deduplicated, PII-masked requests submitted to the Perplexity Decision API during its first 18 days of availability. Because each request can contain multiple questions about the same state, the sample comprises 2,318,887 individual questions. All requests were included with explicit user permission for research.

Production requests do not come with ground-truth labels, so we use frontier LLM judges to generate reference answers and measure each model’s agreement with them. We also group the requests into 10 task categories, including information extraction, domain classification, sentiment analysis, intent and routing, relevance ranking, content moderation, quality and compliance review, entity matching, business analysis, and an “Other” category. This breakdown shows how performance varies across real-world use cases.

The task mix is uneven: information extraction accounts for 425,402 questions, domain classification for 409,564, and sentiment analysis for 344,158. Together, these three categories make up 50.8% of the sampled questions.

Decider v1.3 achieves the highest agreement with the judges at 93.98%, compared with 93.68% for Decider v1.2 and 93.35% for Jev 1.14. Decider v1.3 and Jev 1.14 are closely matched on domain classification and sentiment analysis, with gaps of 0.34 and 0.07 percentage points, respectively. Jev leads by 0.69 points on relevance ranking, while Decider v1.3 leads by 0.93 points on information extraction (the most common task in the sample) and by 1.21 points on intent and routing.

Expand

The chart below compares the reported cost of running the same one million production requests. Each request can contain multiple questions, so the unit here is requests, not individual questions.

Expand

Robustness

We evaluated label sensitivity on 372 multiple-choice questions, making 4,788 requests per model. We reordered options, reassigned labels to answer texts, and renamed labels while preserving each question’s meaning, then checked whether the model’s semantic answer changed. Lower flip rates indicate greater stability.

Decider v1.3 changed its answer on just 0.27% of questions under each transformation, compared with 2.69–4.84% for Jev 1.14. We also repeated identical inputs to measure serving variability: Decider v1.3 showed no changes, while Jev 1.14 changed on 1.88% of questions.

Expand

How we use Decider

Decider is built for workflows that need fast and structured decisions. In this section we explore how to leverage them to produce meaningful impact on the way we work. Here are some that stood out.

Self-improving security rules

Internally, we are already using Decider to improve our security policies.

The activity of our agents is recorded and a series of rule-based checks are performed to identify any problematic behavior. A rule-based approach means that we can monitor our agents in real-time with negligible latency added. Offline, we use Decider to look at samples from these recorded agent logs. In the case that no problematic behavior was flagged, Decider checks if there is something that our set of rules missed; and in the case that an alarm was raised, Decider checks whether this was a false alarm. Any records where our rule set fails in either case are sent to an agent for triage and a fix to the rules is proposed, to be approved by a member of our security team.

In one day, Decider read 1 billion tokens across 7,000 agent sessions and identified flaws that led to 14 rule fixes, 6 of which were approved and shipped within two days. Replayed on a week of logs, the fixes cut our busiest rule's volume by 52%, and the widened rules caught about 40% more legitimate actions to investigate.

Expand

Monitoring RL rollouts

When we post-train models with reinforcement learning, aggregate metrics such as a terminal reward do not explain why a rollout (an individual attempt at a task) succeeded or failed. It can lose reward unfairly because a tool service or the sandbox failed, or receive undeserved reward if the evaluator overlooks an unsupported claim, such as a made-up value that no tool returned.

We use Decider to monitor rollouts for these cases. Code first identifies parts of a rollout that warrant closer inspection: a failed tool call and the surrounding reasoning, code that ran without errors, or the final answer. Decider then answers multiple-choice questions about each part, such as what failed in a step (a tool service, an external site, the sandbox, the harness, or the agent’s own code) and whether the agent interpreted the result correctly. We bundle all questions about a given part into a single request and store the probabilities rather than a verdict, so we can adjust thresholds later without querying Decider again. The monitor only observes and diagnoses: it never changes a reward or stops a run.

Expand

Page quality filtering

When crawling web pages to index and serve to users, many pages are too low quality to be of any use. This includes anti-bot walls, CAPTCHAs, and error pages, as well as correctly loaded web pages that just don’t contain any helpful information, such as spam, SEO filler or navigation-only pages. These low-quality pages need to be identified and filtered out of our index.

Using Decider, we can judge the quality of pages as we crawl them. In the same forward pass, we perform a suite of other classification tasks such as identifying the domain and intended audience of the content and whether it contains harmful, illegal, or personally identifiable information. Compared to using an LLM judge, Decider is considerably faster and cheaper.

Expand

Beyond these internal uses, Decider also fits workflows such as the following.

Visual QA

Because Decider reads images, it also works as a visual checker, reviewing style changes to web pages or apps and identifying faults. It can review page screenshots for clipped text, overlapping elements, broken layouts before a change ships, or flag charts whose design could mislead a reader or break a brand guide.

Expand

Browser navigation

Agents can use Decider for the small decisions inside larger tasks. A browser agent can send a screenshot at each step and receive detailed information about the state of the page and the expected outcome of each possible action.

Expand

Planning research tasks

Users’ questions can greatly vary in their complexity, as does the depth of search that needs to be performed to find an answer. Decider can estimate this before search begins, allowing for fewer resources to be spent on simple questions.

Expand

Getting started

Decider returns probabilities. The appropriate decision threshold depends on the application. Label frequencies and the relative costs of false positives and false negatives vary across use cases. We therefore encourage users to tune thresholds on a small, representative labeled development set rather than assume that a fixed cutoff will work equally well across tasks.

We recommend using Decider v1.3 for nuanced judgments, detailed rubrics, and workflows where errors are costly. Choose Decider Flash v1.3 for high-volume routing, labeling, and filtering where latency and cost are the main constraints. Evaluate both models on representative examples from your workload, then choose the model that meets your quality, latency, and budget requirements.

Both models are available as downloadable weights on Hugging Face under Apache license, through OpenRouter, and the Perplexity Decisions API.

Citation

@online{perplexity2026PerplexityDecider,
  author = {{Perplexity} and Antoine Chaffin and Jonas Golde and Louis Milliken and Sarah Eslami and Markus Krimmel and Nandor Licker and Aleksandr Nikolenko and Ron Heichman and Bo Wang and Jarrod Bryan and Jeremy Yang and Denis Yarats},
  title = {{Perplexity Decider}},
  date = {2026-11-12},
  url = {https://www.perplexity.ai/hub/blog/perplexity-decider}
}