Browser Agents and Long-Horizon Web Tasks

A browser agent is a long-horizon agent that reads untrusted content and takes consequential actions. Each page it visits is input the user did not write, and each form on that page is a possible command channel. The 2025-2026 attack corpus shows this is not hypothetical. Tainted Memories lets an attacker bind malicious instructions to the agent's memory through a crafted page. HashJack hides commands in URL fragments the agent visits. Perplexity Comet hijacks landed with a single click.

The defensive picture is uncomfortable. OpenAI's head of preparedness said indirect prompt injection "is not a bug that can be fully patched." The attack lives on the boundary between the agent reading and the agent acting, and that boundary is architecturally fuzzy: any token the model reads could, in principle, be treated as an instruction.

This lesson names the attack surface and the benchmark landscape (BrowseComp, OSWorld, WebArena-Verified), then models a minimal indirect-prompt-injection scenario so you can reason about real defenses in Lessons 14 and 18.

The 2026 landscape, in one paragraph per system

ChatGPT agent (OpenAI). Launched July 2025. It unifies Operator (browsing) and Deep Research (multi-hour research), and the standalone Operator was shut down August 31, 2025. It holds SOTA on BrowseComp at 68.9%, with strong numbers on OSWorld and WebArena-Verified.

Claude Sonnet + Vercept (Anthropic). Anthropic's Vercept acquisition focused on computer-use capabilities and moved Claude Sonnet on OSWorld from <15% to 72.5%. Claude Computer Use ships as a tool API.

Gemini 3 Pro with Browser Use (DeepMind). Browser Use integration ships computer-use controls, and FSF v3 (April 2026, Lesson 20) tracks autonomy in the ML R&D domain specifically.

WebArena-Verified (ServiceNow, ICLR 2026). Fixes a well-documented problem: the original WebArena had a ~11.3% false-negative rate, meaning tasks marked failed that were actually solved. The Verified release re-grades with human-curated success criteria and adds a 258-task Hard subset (ICLR 2026 paper, openreview.net/forum?id=94tlGxmqkN).

BrowseComp vs OSWorld vs WebArena

These measure different axes. A high BrowseComp score says the agent finds facts, not that it can book a flight. OSWorld is closer to "does it work on my desktop," and WebArena-Verified is closer to "can it finish a flow." For a production decision, pick the benchmark that matches your task distribution.

The attack surface, named

1. Indirect prompt injection. Untrusted page content contains instructions, the agent reads them, and the agent executes them. Public examples: 2024 Kai Greshake et al., 2025 Tainted Memories paper, 2026 HashJack (Cato Networks).
2. URL fragment / query injection. The #fragment or query string of a crawled URL contains commands. It is not rendered visibly but is still inside the agent's context.
3. Memory-binding attacks. A page instructs the agent to write a persistent memory (Lesson 12 covers durable state). In the next session the memory fires the payload with no visible trigger.
4. CSRF-shaped attacks on authenticated sessions. Tainted Memories class: the agent is logged in somewhere, and the attacker's page issues state-changing requests the agent executes with the user's cookies.
5. One-click hijack. A visually innocuous button carries a payload the agent follows. Comet class.
6. Content-Security-Policy holes in the agent's host surface. The rendering and tool layers can themselves be attack vectors, and the browser-in-a-browser-agent stack is wide.

Why "not fully patchable"

The attack is isomorphic to the agent's capability. The agent has to read untrusted content to do its job, content it reads could contain instructions, and instructions it follows could be misaligned with what the user actually asked for. Defenses (trust boundaries, classifiers, tool allowlists, HITL on consequential actions) raise the cost of the attack and reduce its blast radius. They do not close the class.

This is the same reasoning pattern as Lob's theorem (Lesson 8). The agent cannot prove the next token is safe. It can only set up a system where unsafe tokens are more detectable.

Defense posture that actually ships

• Read / write boundary. Reading is never consequential. Writing (submitting a form, posting content, calling a tool with side effects) requires fresh human approval if the initiating content came from outside the trust boundary.
• Tool allowlist per task. The agent can browse, but it cannot initiate a wire transfer unless that tool was explicitly enabled for the task. Lesson 13 covers budgets.
• Session isolation. Browser agent sessions run with scoped credentials only: no production auth and no personal email. Logs of every HTTP request are retained for audit.
• Content sanitizer. Fetched HTML is stripped of known-bad patterns before it is concatenated into the model context. This reduces the easy attacks and does not stop sophisticated payloads.
• HITL on consequential actions. Propose-then-commit pattern (Lesson 15).
• Canary tokens on memory. If a memory entry fires, the user sees it (Lesson 14).

code/main.py models a tiny browser-agent run against three synthetic pages. One page is benign, one has a direct prompt-injection blob in visible text, and one has a URL-fragment injection that is not visible but sits inside the agent's context. The script runs each page against four defense configurations (none, sanitizer, read/write boundary, both) and shows what a naive agent would do, what the read/write boundary catches, what the sanitizer catches, and what the sanitizer misses that only the boundary catches.

outputs/skill-browser-agent-trust-boundary.md scopes a proposed browser-agent deployment: which trust zones it touches, what it is authorized to write, and which defenses must be in place before the first run.

1. Run code/main.py. Identify which attack the sanitizer misses and which defense still catches it. Then check whether any page gets past the read/write boundary, and name the assumption the toy makes to produce that result.

2. Extend the sanitizer to detect one class of HashJack-style URL-fragment injection. Measure the false-positive rate on benign URLs with legitimate fragments.

3. Pick one real browser-agent workflow you know (e.g., "book a flight"). List every read and every write. Mark which writes need HITL and why.

4. Read the WebArena-Verified ICLR 2026 paper. Identify one category of task where the original WebArena's scoring was unreliable and explain how the Verified subset resolves it.

5. Design a memory canary for a browser-agent setting. What would you store, where, and what triggers the alarm?

• OpenAI: Introducing ChatGPT agent: merge of Operator and deep research; BrowseComp SOTA.
• OpenAI: Computer-Using Agent: the Operator lineage and the architecture that became ChatGPT agent.
• Zhou et al.: WebArena: the original benchmark.
• WebArena-Verified (OpenReview): ICLR 2026 fixed-subset paper.
• Anthropic: Measuring agent autonomy in practice: includes attack-surface discussion for computer-use agents.