BrowseSafe++: Building stronger defences against real-world prompt injection

Agents that can browse the web, such as the assistant built into Comet or Computer, need to interact with web pages, reading content and using it to perform actions on behalf of their users. This makes them susceptible to indirect prompt injection, where an attacker places malicious text on a webpage. If the agent follows these instructions, it can return a misleading answer or take an action the user didn’t request.

In December 2025, we released BrowseSafe, a benchmark and detection model for this threat. We built it by adding synthetic attacks to realistic webpages agents were likely to encounter. Over the past year, attacks have evolved to be more sophisticated and look more like ordinary page content. Instead of using tell-tale phrases such as ‘ignore previous instructions’, they often mimic content the agent is likely to trust, such as an official notice from the site or a note about user preferences.

Motivated by these changes, we built an updated benchmark, BrowseSafe++, to measure whether current injection detectors catch these attacks and whether agents can resist them. It contains 38,675 attacked and clean page pairs, primarily built around injections collected in the real world. For each page, we rewrite the surrounding content to protect privacy but keep the attack verbatim. Every page comes in the four formats browser agents read: HTML, Markdown, accessibility trees, and screenshots. We used a subset of the benchmark to train BrowseSafe-Guard, a pair of 0.8B-parameter detectors: a text detector and a multimodal detector. The two Guard detectors identify injections more accurately than the prompted models we tested, including frontier models. They are also faster, cheaper, and flag far fewer ordinary pages by mistake.

We’re releasing BrowseSafe++ as an open benchmark. The project website links to the dataset, SDK, research paper, and a public leaderboard. Any team can score its detector against test pages whose labels are not provided and submit the labels for grading and possible inclusion in the leaderboard. To evaluate and submit your results, see Getting started.

Building a benchmark from observed injections

BrowseSafe++ is built from prompt injections observed on real websites. Between September 2025 and September 2026, Comet’s prompt injection detector flagged 1.15 million injections on pages its agent read. We built 32,187 pairs around those injections and added 6,488 pairs from the original BrowseSafe benchmark, for a total of 38,675 pairs. Each pair holds both an attacked page and its clean twin, which is the same page without the injection.

What injections look like in the wild

Two examples from the benchmark show what these injections look like in practice.

In one, a fake purchase request encourages the agent to take an unauthorised action. Comet’s detector flagged this block of English text on a Russian-language developer documentation page in October 2025:

The text opens and closes with orders and fills the middle with ordinary token marketing. It claims to come from the agent’s own developer and ends with an authorisation code to make the purchase look approved.

In another, a forged conversation aims to disrupt the agent’s task. This text was found on a code repository page in March 2026:

The text is designed to look like a conversation between a user and an agent: <|im_start|> and <|im_end|> are markers some chat models use to separate the turns of a conversation. An agent that fell for this attack would respond to the forged user query and reply with the attacker’s phrase.

We observe similar patterns across the flagged injections:

• They emulate trusted sources. Both examples make the page text sound like a source the agent trusts: its developer in the first, its own conversation in the second. Injections can also pose as the site itself, mimicking security checks and policy notices, or as the user, putting forth fake preferences. About two in three are written in formal language.
• Many are written in different languages. In 24.4% of pairs, the injection uses a language that is different from the rest of its page. This typically takes the form of an English instruction on a non-English page, as in the first example. Such attacks are clearly addressed to an agent that reads every language and is usually native in English. Language switches also make detection harder: in our original BrowseSafe evaluation, multilingual attacks were among the hardest to catch.
• Most want actions, not words. More than four in ten push the agent towards an action the user never asked for, like the token purchase in the first example, and about a third try to make it drop the user’s task, as in the second example. Only about 3% try to plant false information in the answer.
• They often ride on content anyone can post. An attacker does not need to control a site. A forum post or a shared document on ordinary sites is enough, and more than 23,000 injections in BrowseSafe++ arrived this way. A trusted domain can still be utilised for injecting an attacker’s text.

The injections present in the final benchmark span 12 types of websites, 24 attack types, 6 attacker goals, and 18 placements on the page.

Expand

From a flagged page to a benchmark pair

We cannot use flagged pages as they are in the benchmark, because they may contain personal information from the user the agent was working for. Turning a page into a shareable benchmark pair requires three main stages:

1. Source the injection: We collect the flagged pages, remove duplicates, and mask personal information with PII-Tracer. Then, we confirm that the page contains an injection; specifically, we require a panel of models to agree that it contains an instruction that tries to redirect an agent away from the user’s intended task.
2. Build the pair: An agent removes the injection text from each page and rewrites the surrounding page content, replacing details such as names and brands with fictional ones. A second agent checks the rewritten page for leftover real-world content and sends failures back with feedback. Human reviewers inspect a random 10% of pages; problems they find are fixed in the pipeline, and the fixes are applied across the corpus. The injection is reinserted into the rewritten page to create the attacked copy and left out of the clean copy. The two pages share the same task, expected result, and ordinary instructions.
3. Verify: Agents read pages in different forms, so we render each page’s HTML as Markdown, an accessibility tree (a structured list of a page’s interface elements and their labels), and a screenshot. Another agent checks that the injection appears in all four views of the attacked page and that the clean page is judged benign in every view. If the injection is missing in one of the views, we move it to a different spot on the page and render again, ensuring that the injection occurs in the final attacked page in all views. A final set of automated checks corrects any pairs that still carry traces of their source, such as real hostnames or brand names.

Expand

BrowseSafe-Guard: Fast and precise detectors for every view

A useful detector needs to check every page quickly and cheaply. It also needs to be accurate on both attacked and clean pages. High precision is especially important, since most pages are safe; even a relatively low false positive rate would make most alerts false alarms. Prompted general-purpose models often fall short on precision and are too slow and costly to run on every page.

A detector trained on attacked and clean pages collected separately could learn shortcuts that happen to separate the two sets, such as a page’s topic or layout. Such a detector, however, doesn’t generalise well. Guard instead trains on pairs whose only difference is the injection itself, which helps improve detector precision. For each pair, Guard reads the attacked page and its clean twin separately and gives each a score for how likely it is to contain an injection. Two training objectives are used to update the model: the standard classification objective and a ranking objective that penalises Guard unless the attacked page scores higher than its twin by at least a fixed margin. Intuitively, the classification objective teaches Guard what decision to make, and the ranking objective ties that decision to the injection itself and ignores benign distractors. This trains Guard to respond to the injection itself, rather than the surrounding page content.

Guard consists of two detectors, each fine-tuned from Qwen3.5-0.8B. The text detector reads one text view of a page: either HTML, Markdown, or accessibility tree. The multimodal detector reads a full-page screenshot. In deployment, Guard reads a page in a single forward pass and outputs an injection score, and the page is flagged if the score passes a threshold. Since Guard has only 0.8B parameters and outputs a single score, it’s fast and cheap enough to run on every page.

Benchmarking performance

We compared Guard with 18 models prompted to detect injections on a held-out subset of the benchmark. We report four metrics, measured on BrowseSafe++:

• Recall is the share of injected pages that the detector flags, which measures how many attacks it catches, so higher is better.
• The false-positive rate (FPR) is the share of clean pages that the detector flags by mistake, which measures how often it raises false alarms, so lower is better.
• Precision is the share of flagged pages that really contain an injection, which measures how often an alert is correct, so higher is better.
• F1 is the harmonic mean of recall and precision; it’s high only when a detector catches most attacks, and few of its flags are false alarms.

We use F1 as the primary metric, since a good detector should have high precision and recall. All four metrics are averaged across the three text views.

Expand

Expand

Model comparisons

The small open-weight models we tested landed at the bottom. Mistral NeMo and Qwen3 30B-A3B each flag about half of the clean pages, while Llama 3.1 8B rarely flags anything, missing most attacks. Even the strongest model of the group, GPT-OSS 20B, only reaches an F1 of 70.97%. DeepSeek V3.1, a much larger open-weight model, does slightly better, with an F1 of 75.23%.

Lightweight API models including GPT-5.4 nano, Gemini 3.1 Flash-Lite, and Claude Haiku 4.5 catch more attacks, with recall between 85% and 97%. However, the cost is a high share of false alarms; Gemini 3.1 Flash-Lite has the lowest FPR of the group at 35.2%, and Haiku’s FPR is nearly 69%. As a result, the maximum F1 score of the group is 80.9%, attained by Gemini 3.1 Flash-Lite.

The remaining models are stronger general-purpose models, reaching F1 scores between 85% and 92%. However, they still face trade-offs between recall and FPR; models that lean towards caution block legitimate pages, and models that raise flags less frequently miss attacks. Claude Sonnet 5 has the highest recall of any detector at 98.73% but an FPR of 24.65%. Gemini 3.1 Pro and GPT-5.5 reach recalls of 97.78% and 97.19%, with FPRs of 19.80% and 20.44%, respectively. GPT-6 Astra has the highest overall accuracy, with a recall of 96.42% and an FPR of 14.41%. Decider comes second, with a recall of 96.02% and an FPR of 17.44%. Claude Opus 5.5 has the lowest FPR at only 4.21%, but its recall of 77.71% means it misses more than one in five attacks. Claude Fable 5.1 looks strong on the pages it answers, with a recall of 89.31% and an FPR of 7.92%, but it refuses to answer on about 30% of pages, which the ‘(all)’ row counts as flags.

Expand

Guard advances the Pareto frontier, achieving a better trade-off between missed attacks and false alarms. It reaches an F1 of about 95% and ranks first in every text view. It flags only about one clean page in 20, roughly a third the rate of Astra, the most accurate prompted model. This difference comes from our training; the stock Qwen3.5-0.8B that Guard is built on, prompted the same way as the other models, only reaches an F1 of 64% and flags nearly three in four clean pages. In the screenshot view scenario, we compare Guard’s multi-modal detector with four image-capable models. Given full-page screenshots, Guard scores about 93% F1, against about 79% for the best of the other models we evaluated.

Recognising an attack is not the same as resisting it

Intuitively, a model that can correctly identify an injection should also ignore that injection when it encounters the page while completing user tasks. However, our experiments show that strong detection scores don’t guarantee that a model will resist an attack. We gave each model the same attacked page twice, once to judge whether it contained an injection and once to carry out the user’s task without seeing the previous verdict. A separate judge checked whether it followed the injection. Every model tested still followed some injections it had flagged, from 1.64% of flagged pages for GPT-6 Astra to 24.73% for Llama 3.1 8B.

Expand

Detection scores don’t reliably predict differences across models. Claude Sonnet 5 and GPT-5.5 detect injections about equally well, with respective F1 scores of about 88% and 89%. However, Sonnet 5 followed an injection it had flagged on 19.74% of pages, ten times as often as GPT-5.5 at 1.97%.

A deployable system for injection defence

These results suggest that there is value in delegating the decision to block a page to a special-purpose model, instead of relying on the model carrying out the user’s task. A separate detector that screens each page before the agent acts provides an extra layer of defence and has no task to hijack or tools to misuse.

Guard is built to be that detector. Beyond having the highest F1 and accuracy of all models we tested, its small size makes it cheap enough to run on every page. On average, Guard used about 0.15 seconds of GPU time per text page. On a rented H200, we estimate a cost of $0.18 per 1,000 pages, compared to $18.95–180.61 for the frontier API models at list prices. This makes frequent screening much cheaper in our comparison while retaining high detection scores.

The figure below plots each detector’s accuracy against its time per page and its cost per 1,000 pages, both on log scales. Guard sits alone in the upper-left corner of both panels: it wins on accuracy, speed, and cost. Every prompted model takes about a second or more per page, and the two most accurate, GPT-6 Astra and Claude Fable 5.1, cost over $100 per 1,000 pages.

Expand

Testing defences against adaptive attacks

The BrowseSafe++ benchmark has a finite set of injections. However, a real attacker can mount an adaptive attack; when an injection fails, the attacker can rewrite it and try again. As a result, an agent that resists the original injection could still be susceptible to future attacks. Moreover, models are trained on data collected from the internet, so they may learn to resist most of its injections; since we are releasing our benchmark publicly, it will likely saturate and fail to differentiate models well. To address both problems, we built BrowseSafe-Adapt, a testing environment in which an attacker model can rewrite injections. Each test has three stages:

1. Set up the case. A case groups a BrowseSafe++ entry with a target agent. The BrowseSafe++ entry includes a scenario with a user query and task, an attacked page, and the attacker’s desired outcome. The agent works in a sandbox that provides the tools and synthetic data required to execute the task. A separate verifier model evaluates whether the attack achieved its goal.
2. Run an attempt: We insert the current injection into the page and let the agent carry out the task. The verifier checks the agent’s answer and any changes it made to the sandbox to determine whether the attack succeeded. The first attempt uses the injection from the original BrowseSafe++ entry.
3. Revise and repeat: If an attack fails, we archive the attempt, and a critic model reviews what went wrong. An attacker model uses that feedback to rewrite the injection; it can change the attack’s wording, formatting, or position on the page. The attack goal, success criteria, and surrounding page content stay the same. The next attempt uses the updated attack. This process is repeated until the attack is successful or the attacker reaches a set attempt limit.

Expand

We tested seven agents, each built on a different model: Claude Sonnet 5, Claude Haiku 4.5, GPT-5.5, Gemini 3.1 Pro, Gemini 3.5 Flash-Lite, and the open-weight models DeepSeek V3.1 and Kimi K2. Each agent faced the same 1,024 tasks, with up to 64 attempts per task. On the first attempt, attack success was low across all seven agents. The original injections succeeded on at most 4% of the evaluated tasks against any agent. Under the adaptive framework, attack success rose dramatically. The attacker succeeded on at least 79% of tasks against five of the seven agents. Even the strongest agents were susceptible: attacks still succeeded on about two in three tasks against GPT-5.5 and about one in three against Claude Sonnet 5.

The difference between agents is largest when the attacker’s objective is to make the agent issue a particular tool call. For tasks with this kind of objective, Sonnet 5 gave way on about 8% of tasks and GPT-5.5 on about 29%. The other five carried out the attacker’s objective on more than half of their tasks.

We also ran BrowseSafe-Adapt against the Gemini 3.5 Flash-Lite agent with Guard screening each page first. Guard blocked 94.5% of the original injections and nearly doubled the average number of attempts the attackers needed, from 9.7 to 17.0. Given all 64 attempts, however, the attacker still succeeded on 84.2% of tasks (93.6% without Guard), almost always with an injection that Guard didn’t flag. This matches the view we laid out when we released BrowseSafe: no detection model is a silver bullet, so a detector has to sit inside a layered defence.

Expand

Conclusion

Browser agents are often susceptible to indirect prompt injection, where attacks hidden on ordinary pages seek to interfere with the agents’ ability to carry out user tasks. The dangers only grow over time, both as agents gain access to more data and tools, raising the potential stakes of attacks, and as attackers can revise their injections to be more effective.

Our BrowseSafe suite addresses these threats. The BrowseSafe++ benchmark contains 38,675 webpage pairs built primarily from real injections, enabling more reliable training and evaluation for detectors. The BrowseSafe-Guard detectors, trained on these pairs, enable highly accurate, fast, and cheap detection, making it possible to scan pages before an agent acts on them. The BrowseSafe-Adapt framework enables testing models against revised injections, highlighting vulnerabilities that might not be apparent from single attempts using fixed injections.

Together, we hope that these tools make it easier for researchers and engineers to find failures and improve defences against web-based prompt injection attacks. We are also continuously evaluating and improving Comet and Computer’s built-in defences to keep our users safe.

Getting started

1. Install the SDK and get access to the data. The SDK installs from GitHub:

The dataset is gated. Request access once on its Hugging Face page, then log in with hf auth login.

2. Submit to the leaderboard. Evaluate your own detector on BrowseSafe++’s test split:

This writes one decision per page for each view you list; list all three text views to get an average on the text leaderboard. Sign in to the leaderboard Space with your Hugging Face account and upload predictions.jsonl. The Space returns F1, accuracy, false-positive rate, and the other metrics for each view and, if you choose, publishes the result on the leaderboard. Each account (at least 60 days old) gets one scored submission per UTC week. The test pages are for evaluation only: do not train on them or redistribute them. The submission guide covers the file format and the publication rules.

Citation

@online{perplexity2026BrowseSafePlusPlus,
  author = {{Perplexity} and Yuetian Chen and Kaiyuan Zhang and Yuntao Du and Thomas Li and Mark Tenenholtz and Kyle Polley and Jerry Ma and Denis Yarats and Ninghui Li and Jarrod Bryan and Claire Beamer and Jeremy Yang},
  title = {{BrowseSafe++: Building stronger defenses against real-world prompt injection}},
  date = {2026-10-28},
  url = {https://www.perplexity.ai/hub/blog/browsesafe-plus-plus}
}