First: does PII detection actually make agents too slow?

Not necessarily.

But there is no useful universal number for "the latency of anonymization". Different detection layers operate on completely different scales.

Regex and simple deterministic checks can run in sub-millisecond or low-millisecond territory. A local transformer NER model may take tens of milliseconds. A remote PII service can easily push the operation into the hundreds of milliseconds once network latency is involved.

One 2026 gateway analysis, for example, uses illustrative figures of below 2 ms for regex and entropy checks, around 35 ms P50 for local NER and around 180 ms P50 for an external PII API, while explicitly warning that the actual values depend heavily on hardware, model size, batching and network conditions.

This is why "PII protection adds 100 ms" is almost meaningless without an architecture attached to it.

The more useful question is:

Where are you putting the expensive detectors?

Running an LLM-based privacy check before every LLM call is obviously going to hurt. Running deterministic rules plus a compact encoder locally is a very different proposition.

Roblox provides a nice reality check on what optimized inference can look like. Its XLM-RoBERTa-Large PII classifier handles peaks above 200,000 queries per second while remaining below 100 ms P90 — although that is a highly optimized GPU-backed production system solving Roblox's specific classification problem, not an off-the-shelf latency guarantee.

So yes, privacy has a cost. It does not follow that the cost has to dominate the request.

Agents change the multiplication factor

The harder problem is that agents don't make one request.

A single user action might cause:

  • an LLM call to decide which tool to use,
  • a database or API lookup,
  • another LLM call to interpret the result,
  • a retrieval operation,
  • another tool call,
  • and finally one or more synthesis calls.

Now imagine that PII enters on the first step. Maybe the user writes:

Can you check what happened with the account for maria@example.com?

You redact it before the first model call. Fine.

But the agent then queries your CRM, which returns Maria's full name, billing address and customer identifier. You have sensitive data again. The original sanitization step has done nothing to protect that new information.

This is why an agent cannot have one privacy boundary at the user input. It needs boundaries around data movement.

The important unit is the hop

Every time data moves from one trust domain to another, I would treat it as a potential privacy boundary. That might be:

user → agent
tool → agent
retriever → model
model → external tool
agent → log
agent → tracing platform
agent → model provider

The exact boundaries depend on the architecture. The principle doesn't.

A system that carefully sanitizes the original prompt but then sends raw CRM output to a model provider has not solved the problem. It has moved the leak one step deeper into the graph.

Observability is one of the easiest places to get this wrong

There is an uncomfortable side effect of building more sophisticated agent systems: we record more of them. Prompts. Tool inputs. Tool outputs. Intermediate state. Exceptions. Traces. Latency spans. Model responses.

All of this is incredibly useful when an agent starts doing something inexplicable in production. It is also a very effective way to create a second copy of every sensitive value the agent has ever seen.

This is the sort of privacy problem that does not show up in the architecture diagram if the diagram only contains "User → Agent → LLM". The telemetry stack is part of the system. So is the vector database. So is the replay system used for evaluations. So is whatever platform your engineers use to inspect traces.

If raw PII can reach any of them, they belong in the threat model.

Sanitizing early is still the best default

For data that does not need to leave a trusted environment in its original form, I prefer transforming it as close to the source as possible. Not because every system must redact absolutely everything. Because the number of places where you have to remember to protect the data decreases dramatically.

If:

maria@example.com

becomes:

[EMAIL_17]

before it enters the rest of the agent graph, every downstream component inherits the safer representation. That is much easier to reason about than asking twelve independent services to remember that email addresses are sensitive.

But simple redaction creates its own problems.

Agents often need identity without needing the identity

Imagine an agent working on this request:

Move Alice's meeting with Bob and send Alice the new time.

Replace both names with [PERSON] and you have destroyed useful information. The model does not need to know that Alice is Alice. It does need to know that Alice and Bob are different people and that the first person is also the recipient of the notification.

That suggests a better transformation:

Move [PERSON_1]'s meeting with [PERSON_2] and send [PERSON_1] the new time.

You have removed the original names while preserving the graph structure the model needs to reason over.

For agentic systems, this kind of referential consistency is extremely important. It is one reason reversible pseudonymization or surrogate substitution is becoming more interesting than simple black-box redaction. Replacing real identifiers with type-consistent surrogates preserves more semantic structure for the model while the original PII stays local, which is exactly the trade you want.

There is a catch. The mapping:

[PERSON_1] → Alice Smith

is sensitive data itself. You have not made the privacy problem disappear. You have concentrated it into a smaller, controllable component. That is still useful, but it needs encryption, access control, sensible retention and strict scoping.

And this is where anonymization becomes a dangerous word

If the system can restore [PERSON_1] back to Alice, the data is generally pseudonymized rather than truly anonymous. The distinction is not academic.

The European Data Protection Board explicitly distinguishes pseudonymization — reducing the ability to link data to a person while preserving that possibility under controlled conditions — from anonymization, where the link is effectively removed.

That means reversible identifiers do not magically move a pipeline outside the GDPR. They are a security and data-protection measure within it.

This matters because teams sometimes describe any value that no longer contains a readable name as "anonymous". A UUID can still represent a person. A stable hash can still represent a person. CUSTOMER_82371 can still represent a person. The important question is whether the data can reasonably be linked back to them.

The AI Act does not replace that problem, and the headline fine gets misread

There is also a tendency to put every AI-related compliance question under the EU AI Act now. For this particular architecture, I think that is misleading. The privacy problem we're discussing is still primarily a data-protection problem. GDPR applies to the processing of personal data regardless of whether the software touching it happens to contain an LLM.

The AI Act adds a separate set of obligations. And the timing is particularly interesting right now. Article 50 transparency obligations became applicable on 2 August 2026. They include transparency requirements around certain human-AI interactions and AI-generated or manipulated content.

At the same time, the AI Omnibus that entered into force on 27 July 2026 pushed major high-risk-system deadlines further out. For Annex III high-risk systems, the relevant rules now start applying on 2 December 2027. For high-risk AI embedded in products covered by Annex I, the date is 2 August 2028.

So two things can be true at once: some AI Act obligations are already very real, and some of the obligations people associate most strongly with "AI Act compliance" are not yet applicable.

The headline everyone remembers is: €35 million or 7% of worldwide annual turnover. That number is real, but it applies to the most serious category: violations of the prohibited AI practices in Article 5. There is an important detail that is often stated incorrectly. For an undertaking, Article 99 sets that maximum at the higher of €35 million or 7% of worldwide annual turnover. For SMEs, including startups, the regulation instead caps the fine at the lower of the fixed amount and percentage.

Other violations have lower ceilings. For example, Article 50 transparency violations fall within the category whose ceiling is €15 million or 3% of worldwide turnover for undertakings, subject again to the SME rule. The law also requires penalties to be proportionate, and the authority considers factors including company size, cooperation, mitigation, responsibility and whether the infringement was intentional or negligent.

So the social-media version — "one mistake and your startup gets a €35M fine" — is not a useful model of how the regulation actually works. Neither is pretending the fines do not matter.

The architecture I would optimize for is therefore not AI Act compliant

I would optimize for something more durable: know where personal data exists and control every transition it makes.

In practice that means knowing which events contain personal data, which tools can introduce new data, which identifiers have to stay stable for the model to reason, where the reversible mappings live, what ends up reaching third-party models, what gets written to logs, how long each representation survives, and where the original value can be reconstructed.

This is less glamorous than adding a redact_pii() call to the beginning of an agent. It is also much closer to the real engineering problem.

The interesting part of agent privacy is not the filter

The filter is becoming the easy part. We have regex. We have NER. We have increasingly strong PII-specific encoders. We can use multiple detectors. We can pseudonymize. We can generate surrogates.

The difficult question is what happens after the first detection. Agents move information between components constantly, so privacy controls have to stop being a preprocessing step and become part of the execution architecture.

Once you look at it this way, the design changes. You stop asking: "Did we anonymize the user's prompt?" and start asking: "Can any path through this agent move personal data somewhere it shouldn't?"

That is a much harder question. It is also the one worth solving.