Securing AI Agents: Think in Layers, Not Just Guardrails
How to let an AI agent work with private data without handing it the keys: six independent layers of protection, illustrated with a privacy-first email assistant.
Many teams want an AI agent that works with their private data: email, documents, customer records. Many of them hesitate, for good reason. An agent that can read sensitive information and take actions is only as safe as the system around it.
We recently worked with a privacy-first client on an agent that processes incoming email, identifies what needs attention, and proposes responses.
The challenge wasn’t just making the agent useful. It was making sure that mistakes, malicious instructions, or unexpected behavior couldn’t lead to unrestricted access or data exposure.
Picture an email containing hidden instructions to retrieve confidential information and send it to an external address. The agent is supposed to read that email. It must certainly not obey it.
Our approach is to build multiple layers of protection, each one limiting what the agent can do if another layer fails. Here’s how we think about it.
1. Start with infrastructure isolation
Run agents in isolated environments: virtual machines or microVMs where stronger security boundaries are warranted. Hardened containers may be enough for lower-risk workloads.
Don’t run agents as root. Create dedicated, non-privileged OS users with access only to the files and services they need. An email-processing agent has no reason to touch the host machine’s filesystem, unrelated applications, or administrative privileges.
If an agent needs to execute arbitrary code, that should happen in a separate, tightly controlled sandbox.
The idea is simple: even if the agent is compromised, the blast radius stays small.
Of course, a VM alone won’t stop an agent from misusing the data or credentials it has legitimately been given. That’s why we need more layers.
2. Lock down the network
Keep agent infrastructure on private networks where possible, using Tailscale or a similar private networking solution. Expose only necessary ports, and restrict both inbound and outbound connections.
A VPN alone isn’t sufficient. You also need firewall rules, appropriate network segmentation, and outbound access restrictions.
For example, an email agent might need access to a model API and a narrowly scoped email service. It shouldn’t be able to connect to arbitrary destinations on the internet.
Even with outbound connections restricted, an agent could still leak data through an authorized service if that service allows unrestricted actions. So network restrictions have to work alongside application-level permissions.
3. Apply least privilege at the agent level
This is where the boundaries matter most. Give agents only the tools they need, and restrict what those tools can do.
For the email agent:
- Read emails? Yes.
- Classify, summarize, and suggest responses? Yes.
- Send emails independently? No.
- Access unrelated files or databases? No.
- Create or modify documents? No.
- Execute arbitrary commands? No.
Simon Willison describes a key risk as the “lethal trifecta”:
- Access to private data
- Exposure to untrusted content
- The ability to communicate externally
An email agent naturally has the first two. It reads private email, and that email comes from outside sources that can’t necessarily be trusted. Any message could contain hidden instructions meant to manipulate it.
The third capability isn’t limited to sending email. Any tool or integration that lets sensitive information reach an unauthorized destination can become an exfiltration path. Even something as harmless-looking as creating a document carries risk: if the agent can create a publicly accessible document containing private information and expose its link, an attacker can exploit that.
Giving the agent any way to move private information to unauthorized destinations completes the trifecta. Avoid that combination wherever possible.
One way to do that is to separate agents or services by responsibility. A read-only agent analyzes information and proposes actions; a separate, more privileged component handles authorized changes.
That separation must be enforced through real tool permissions and credentials, not instructions in the system prompt. The email agent should hold a Gmail read-only token, with no permission to create drafts, modify messages, or send email.
A subtle example: Gmail’s gmail.compose OAuth scope permits both creating drafts and sending email. If the reasoning agent only needs to read and draft, giving it that scope introduces unnecessary risk. Instead, the agent gets read-only access, while a separate, trusted, deterministic service holds the write credential and exposes only one operation: create a draft.
4. Separate reasoning from execution
A principle we increasingly favor:
Agents plan and propose. Deterministic code validates and executes.
The agent can decide that an email needs a response and generate a proposed draft. A separate, trusted service then validates the request, checks permissions, and performs the authorized operation.
In the email system, the agent runs in its own VM, with network access restricted to a small set of approved services. It talks to the execution service through a narrowly scoped, authenticated API. It can submit a proposed draft, but it can’t access the execution service’s credentials or invoke arbitrary operations. The execution service independently validates the request, checks authorization, and performs only the permitted action.
This is enforced through code, not the agent’s instructions.
5. Put humans in the loop where it matters
For consequential or externally visible actions, require explicit human approval.
In the email workflow, the agent reads and analyzes messages, then proposes a response. Once a person approves it, a controlled service creates the response in Gmail Drafts, where the user can review and send it.
The user keeps final control over external communication, while still benefiting from automated triage and drafting.

The email workflow: the agent prepares a response, but draft creation and sending stay behind separate, human-controlled steps.
6. Don’t forget data privacy and operational security
A few additional protections matter just as much:
- Credentials. Keep secrets out of the agent’s context. Use narrowly scoped service identities and short-lived credentials where practical.
- Data privacy. Be deliberate about what private data goes to external model providers. Consider data minimization, local inference where appropriate, and providers with suitable data-handling and retention policies.
- Logs and memory. Private information shouldn’t unnecessarily end up in logs, traces, or persistent memory. Untrusted content shouldn’t be able to modify durable instructions or security policies.
- Observability and auditing. Trace agent decisions, tool calls, policy checks, approvals, and outcomes. Watch for unusual access patterns, repeated authorization failures, and unexpected external communication.
- Resource controls. Apply rate limits, execution timeouts, and resource limits.
- Testing. Test against malicious emails, prompt injection, unexpected tool behavior, and attempted data exfiltration.
Putting the layers together

An isolated mail worker, restricted credentials and network paths, and a separate approval service that alone can create Gmail drafts.
The bigger principle
Not every agent needs a dedicated VM, multiple services, or an elaborate approval workflow. The right controls depend on what data the agent handles and what actions it can take.
The risk rises sharply when an agent combines access to private information, exposure to untrusted content, and the ability to interact with external systems.
The goal isn’t to pile up security controls. It’s to create independent boundaries, so that a failure in one layer doesn’t automatically compromise the whole system.
Security shouldn’t depend on an agent consistently making the right decision. It should come from the architecture around it.
We don’t need to make an agent trustworthy enough to give it unlimited access. We need to design the system so that we don’t have to.
Questions to ask before you deploy an agent
If you’re evaluating an agent, whether you’re building it or buying it, these questions surface most of the risk quickly:
- If the agent followed a malicious instruction, what’s the worst it could do?
- Does it combine private data, untrusted content, and any way to send information out?
- Which limits are enforced by credentials and code, and which only by the prompt?
- Which actions need human approval, and who gives it?
- Where does our data go: which model providers, which logs, and for how long?
- How would we know if something went wrong?
Also published on LinkedIn.