Your AI Assistant Doesn’t Need to Be Hacked to Leak Your Data.

Your AI Assistant Doesn’t Need to Be Hacked to Leak Your Data. It Just Needs to Read the Wrong Email.

by Denis Huré,  Managing Consultant, TL Agency & Consultancy

What documented incidents across Microsoft Copilot, ChatGPT and Claude reveal about AI confidentiality, and the controls that actually address it.

The email nobody opened

In January 2025, researchers at Aim Security found that an email sent to a Microsoft 365 Copilot user could cause harm even if nobody opened it or clicked anything. All that had to happen was for the user to later ask Copilot an ordinary work question. Copilot would retrieve the email as background context, follow instructions hidden inside it, and send sensitive data from its context to a server controlled by the attacker (SecurityWeek).

Microsoft rated the flaw, EchoLeak (CVE-2025-32711), as critical and fixed it on its servers before going public. It told Fortune the issue was “addressed before our customers were impacted” (Fortune). No customer lost data, but the experiment showed something new. The weak point was no longer the employee who clicks a bad link. It was the assistant reading on the employee’s behalf.

Boards are asking the wrong first question

For three years, the standard boardroom question about generative AI has been: “Does the vendor train its models on our data?” It is a fair question, and enterprise contracts now answer it in detail. But the documented record points to a different, more pressing question:

What can a stranger make our AI assistant do with the access we have given it?

The difference matters. A model that remembers your data is a storage problem. An agent that can read your mailbox, follow instructions it finds there, and send information outside the organisation is an authority problem. It has been delegated access, and anyone who can put text in front of it may be able to direct it.

Not one of the incidents reviewed for this article shows customer data being used to train a model. Many of them show assistants being steered by content they were only supposed to read.

Why this became urgent in 2026

Two things changed. Assistants gained agency: they now search, browse, run code, open files and call other tools. And the evidence piled up quickly:

• February 2026: Microsoft confirmed that a Copilot Chat bug had been summarising emails labelled confidential, despite data loss prevention (DLP) policies meant to exclude them (BleepingComputer).
  
  
• February 2026: LayerX showed that a Google Calendar invite could lead Claude Desktop Extensions to run attacker code. Anthropic declined to fix it, saying the issue “falls outside our current threat model” (Infosecurity Magazine).
  
  
• June 2026: Varonis disclosed SearchLeak, in which one click on a crafted link made Microsoft 365 Copilot search a victim’s mail and files and leak the results through Bing (Varonis).
  
  
• July 2026: Anthropic fixed a bypass of the protections it had added after an earlier file-exfiltration demonstration in Claude Cowork (Pallas Security).
  
  
• September 2026: Anthropic reported disrupting a state-linked group that used Claude to automate espionage, including bulk-exporting the mailboxes of drone-component manufacturers (Anthropic).

Many organisations are rolling out exactly these capabilities now, often before their security and governance teams have caught up. IBM’s 2025 Cost of a Data Breach study found that 13% of organisations reported breaches of AI models or applications. Of those, 97% lacked proper AI access controls (IBM).

What the record actually shows

Headlines tend to lump these incidents together as “AI data leaks”. They are not the same thing, and mixing them up leads to the wrong controls. The evidence falls into five distinct categories.

1. Real exposure in a live service

The clearest confirmed case is from March 2023. A bug in an open-source Redis library let some ChatGPT users see the titles of other users’ chats. For a nine-hour window, it may also have shown the payment details of 1.2% of active Plus subscribers: names, billing addresses, card type, last four digits and expiry date. Full card numbers were never exposed (OpenAI). There was no attacker, only a software defect, which is why it is instructive. It was an old-fashioned isolation failure inside a new kind of product.

A second confirmed exposure came through OpenAI’s supply chain. In November 2025, an attacker breached Mixpanel, an analytics vendor OpenAI used, and exported names, email addresses and approximate locations of some API platform users. OpenAI said its own systems were not breached and that no chats, prompts, keys or payment data were involved. It then ended its use of Mixpanel (OpenAI).

2. Attacks proven by researchers

This is the largest category, and it covers all three vendors:

• EchoLeak (Microsoft 365 Copilot): a single email that was never opened, fixed in May 2025 (BleepingComputer).
  
  
• ShadowLeak (ChatGPT Deep Research): Radware hid instructions in an email’s HTML. When a user asked Deep Research to review their inbox, it pulled personal data from another email and sent it to an outside URL. The request came from OpenAI’s cloud, so the organisation’s own network and device security tools would never have seen it. Radware reported a 100% success rate in its repeated tests, and OpenAI marked the issue resolved in September 2025 (Radware).
  
  
• Claude Cowork: two days after Cowork launched, PromptArmor showed that a Word document disguised as a workflow “Skill” could make it upload a confidential loan estimate to the attacker’s own Anthropic account. No human approval was needed at any step (PromptArmor). The destination was Anthropic’s own API, a domain the sandbox trusted. Security researcher Johann Rehberger had disclosed the same route in Claude.ai months earlier (The Register).
  
  
• SearchLeak (Microsoft 365 Copilot Enterprise Search): one click, patched in June 2026 (BleepingComputer).

None of these has a publicly confirmed criminal victim. They are proofs, not breaches. But they are proofs against production products, not laboratory toys.

3. Controls that failed without anyone stealing anything

Microsoft’s incident CW1226324 is the most important case here for regulated industries. Copilot summarised confidential-labelled emails in users’ own Drafts and Sent folders, despite DLP policies meant to keep them out of AI processing. Microsoft said it “did not provide anyone access to information they weren’t already authorized to see”. It also acknowledged that the behaviour “did not meet our intended Copilot experience” (BleepingComputer). Microsoft did not say how many customers were affected (TechCrunch).

Nobody outside the organisation saw those emails. But a control that compliance teams rely on failed silently for weeks. If your regulatory position depends on “confidential content never enters the AI pipeline”, that is a material finding.

4. AI used as a weapon against other organisations

Anthropic’s September 2026 report describes an actor whose activity is “consistent with public reporting linking the actor to Midnight Blizzard”. The actor used Claude to scan and break into government and defence targets, steal a drone-vision software kit, and take more than 300,000 national-identity records from a North African government technology authority (Anthropic). The same report describes a criminal group extracting data from thousands of downstream customers of a breached SaaS vendor.

These were not breaches of Anthropic. In the stolen-API-key campaigns, the report states that “Anthropic’s own systems were not compromised”. The lesson for executives is about speed: AI shortens the attacker’s workflow against everyone, including organisations that use no AI at all.

5. Vendors leaking their own material

In March 2026, Anthropic exposed close to 3,000 unpublished website assets through a misconfigured content system (Fortune). Days later, it published around 500,000 lines of Claude Code source code in an npm release by mistake (Fortune). In both cases Anthropic said no customer data was involved. These belong in your vendor due diligence on operational maturity. They are not evidence that customer conversations leaked.

The pattern behind the headlines

Strip away the product names and every demonstrated exfiltration above follows the same chain. Developer Simon Willison calls it the “lethal trifecta”: an AI system that has (1) access to private data, (2) exposure to untrusted content, and (3) a way to communicate externally (Simon Willison).

• EchoLeak: mailbox + inbound email + automatic image fetch.
• ShadowLeak: Gmail + hidden HTML + cloud browsing.
• Cowork: local folder + a shared document + an allowlisted upload API.
• SearchLeak: enterprise index + a crafted link + Bing’s image proxy.

There is an uncomfortable detail here. In Cowork, SearchLeak and EchoLeak, the data left through infrastructure that was trusted on purpose: Anthropic’s own API, Microsoft’s Bing, and a Microsoft Teams proxy (arXiv). Domain allowlists did not stop exfiltration because the attacker’s account sat behind a trusted domain.

The UK National Cyber Security Centre explains why this keeps happening. Language models do not enforce a boundary between instructions and data, so prompt injection “may never be totally mitigated in the way that SQL injection attacks can be” (NCSC). OpenAI makes the same point about its own agent: prompt injection “is unlikely to ever be fully ‘solved'” (OpenAI). OWASP ranks prompt injection as the top risk for AI applications (OWASP).

The debate: overstated risk or structural flaw?

View A: the risk is real but manageable, and often overstated

The strongest version of this argument goes as follows. After more than three years of mass deployment, the confirmed cross-user exposures in this review are few: a 2023 caching bug, plus a breach at an analytics vendor. Every serious research finding was fixed, several before going public. EchoLeak was patched before disclosure, ShadowLeak within about two months, and SearchLeak before Varonis published. None has a confirmed criminal victim. That is how responsible disclosure is meant to work.

On this view, the confidential-email bug is an ordinary software defect with no outside exposure. Every SaaS platform has those. The threat-report cases are misuse of a general-purpose tool, much like attackers using cloud servers or scripting languages. Meanwhile, the cost of standing still is real. Competitors are getting faster, and banning sanctioned tools pushes employees towards unsanctioned ones. IBM found that shadow AI added an average of US$670,000 to breach costs (IBM). An executive could reasonably conclude that the sensible response is a governed rollout, not a pause.

View B: the risk is structural, and we may not see it when it happens

The opposing argument is equally serious. Patches close specific attack chains; they do not close the class. When both the UK’s national cyber authority and OpenAI say prompt injection may never be fully solved, each fix is a reprieve, not a cure. The same pattern reappeared across three vendors in 18 months. In Anthropic’s case, the protection added after Cowork was itself bypassed and needed a second fix (Pallas Security).

The phrase “no known exploitation” also deserves scrutiny. ShadowLeak’s data left from the vendor’s cloud, not the customer’s network, so customer-side tools had nothing to log. Absence of evidence is weaker when the evidence would sit in someone else’s logs. The confidential-email bug was found by Microsoft, not by customers whose DLP policies were being bypassed. And the September 2026 threat report shows capable attackers already using AI agents at scale. The gap between “researchers can do this” and “criminals do this” is closing.

The second debate: who owns the risk?

Anthropic’s response to the Desktop Extensions research crystallises a contractual question boards have barely started to address. Anthropic argued that users “explicitly configure and grant permissions” to local tools, so the security boundary is the user’s configuration (The Register). LayerX countered that a calendar invite should never become code execution, whatever the user installed.

Both have a point. Vendors cannot control every extension a user installs. But business users cannot judge how two tools will combine when an agent decides to chain them. Wherever your contracts and policies settle this, they should settle it explicitly.

Where I land

Both views are right about different things. View A is right that this is not a reason to abandon AI assistants, and that the record shows fast vendor response. View B is right that vendor patching is not a control your organisation owns. The practical conclusion is that the risk should be designed out of each workflow, rather than left to vendors to patch.

Controls that map to the evidence

What not to conclude

• Not that any one vendor is “safer”. This is a selection of public disclosures, not a census. Vendors with more outside research attention will show more findings.
• Not that fixed flaws remain open. The specific chains above are reported fixed, except the Desktop Extensions issue, which Anthropic declined to change.
• Not that a demonstration equals a breach. Researcher success rates describe test conditions, not victims.
• Not that your data trained a model. None of these incidents establishes that.
• Not that the absence of reports means safety. Cloud-side agent activity may not show up in your logs.

Five questions for your next leadership meeting

1. Which of our AI deployments combine private data, untrusted content and an outbound channel, and who signed off on each?
2. Can our AI agents send, upload or run anything without a human confirming it?
3. How do we know our sensitivity labels and DLP rules actually keep content out of AI processing, and when did we last test them?
4. What logs of agent activity in the vendor’s cloud do our contracts entitle us to, and does anyone review them?
5. Who owns the decision when a vendor says a risk “falls outside our threat model”?

How TLA&C can help

TL Agency & Consultancy helps CEOs and leadership teams adopt AI assistants and agents without handing them uncontrolled access to the business. We map where your AI deployments combine sensitive data, untrusted content and outbound channels. We help design governance that allocates risk clearly between vendors, IT, security and business owners, and turn controls such as least-privilege connectors, human approval for consequential actions, outbound restrictions and audit logs into procurement requirements and operating policy. We also prepare board-level briefings that separate demonstrated risk from confirmed harm, so decisions rest on evidence rather than headlines. Our aim is to keep your AI programme moving while giving leadership a defensible answer to the question every regulator, client and board will eventually ask: what can your AI do with our data? Learn more at tlagency.com.

Bibliography

Vendor disclosures and primary research

1. OpenAI. “March 20 ChatGPT outage: Here’s what happened.” 24 March 2023. https://openai.com/index/march-20-chatgpt-outage/
2. OpenAI. “What to know about a recent Mixpanel security incident.” November 2025. https://openai.com/index/mixpanel-incident/
3. OpenAI. “Continuously hardening ChatGPT Atlas against prompt injection attacks.” December 2025. https://openai.com/index/hardening-atlas-against-prompt-injection/
4. Anthropic. “Detecting and countering misuse of AI: September 2026.” https://www.anthropic.com/threat-intelligence-report-september-2026
5. Radware (Babo, Nakibly, Uziel). “ShadowLeak: A Zero-Click, Service-Side Attack Exfiltrating Sensitive Data Using ChatGPT’s Deep Research Agent.” 18 September 2025. https://www.radware.com/blog/threat-intelligence/shadowleak/
6. Varonis Threat Labs (Taler). “SearchLeak: How We Turned M365 Copilot Into a One-Click Data Exfiltration Tool.” June 2026. https://www.varonis.com/blog/searchleak
7. PromptArmor. “Claude Cowork Exfiltrates Files.” January 2026. https://www.promptarmor.com/resources/claude-cowork-exfiltrates-files
8. Pallas Security (Badishi). “Anthropic’s Claude Cowork Improper Network Filtering Allows Exfiltration of Sensitive Files.” 14 July 2026. https://pallassecurity.com/blog/claude-cowork-network-filtering/
9. Reddy, P. and Gujral, A. S. “EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System.” arXiv:2509.10540. https://arxiv.org/html/2509.10540v1

Official records and guidance

1. NIST National Vulnerability Database. CVE-2025-32711. https://nvd.nist.gov/vuln/detail/CVE-2025-32711
2. NIST National Vulnerability Database. CVE-2026-42824. https://nvd.nist.gov/vuln/detail/cve-2026-42824
3. UK National Cyber Security Centre. “Prompt injection is not SQL injection (it may be worse).” December 2025. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection
4. OWASP Gen AI Security Project. “LLM01:2025 Prompt Injection.” https://genai.owasp.org/llmrisk/llm01-prompt-injection/
5. IBM. “IBM Report: 13% Of Organizations Reported Breaches Of AI Models Or Applications, 97% Of Which Reported Lacking Proper AI Access Controls.” 30 July 2025. https://newsroom.ibm.com/2025-07-30-ibm-report-13-of-organizations-reported-breaches-of-ai-models-or-applications,-97-of-which-reported-lacking-proper-ai-access-controls

Reporting

1. SecurityWeek (Kovacs). “‘EchoLeak’ AI Attack Enabled Theft of Sensitive Data via Microsoft 365 Copilot.” 12 June 2025. https://www.securityweek.com/echoleak-ai-attack-enabled-theft-of-sensitive-data-via-microsoft-365-copilot/
2. Fortune. “Microsoft Copilot zero-click attack raises alarms about AI agents.” 11 June 2025. https://fortune.com/2025/06/11/microsoft-copilot-vulnerability-ai-agents-echoleak-hacking/
3. BleepingComputer. “Zero-click AI data leak flaw uncovered in Microsoft 365 Copilot.” June 2025. https://www.bleepingcomputer.com/news/security/zero-click-ai-data-leak-flaw-uncovered-in-microsoft-365-copilot/
4. BleepingComputer (Gatlan). “Microsoft says bug causes Copilot to summarize confidential emails.” 18 February 2026. https://www.bleepingcomputer.com/news/microsoft/microsoft-says-bug-causes-copilot-to-summarize-confidential-emails/
5. TechCrunch (Whittaker). “Microsoft says Office bug exposed customers’ confidential emails to Copilot AI.” 18 February 2026. https://techcrunch.com/2026/02/18/microsoft-says-office-bug-exposed-customers-confidential-emails-to-copilot-ai/
6. BleepingComputer (Toulas). “New attack turned Microsoft 365 Copilot into 1-click data theft tool.” 15 June 2026. https://www.bleepingcomputer.com/news/security/new-attack-turned-microsoft-365-copilot-into-1-click-data-theft-tool/
7. BleepingComputer. “OpenAI ChatGPT payment data leak caused by open-source bug.” March 2023. https://www.bleepingcomputer.com/news/security/openai-chatgpt-payment-data-leak-caused-by-open-source-bug/
8. BleepingComputer. “OpenAI discloses API customer data breach via Mixpanel vendor hack.” November 2025. https://www.bleepingcomputer.com/news/security/openai-discloses-api-customer-data-breach-via-mixpanel-vendor-hack/
9. The Register. “OpenAI dumps Mixpanel after analytics breach hits API users.” 27 November 2025. https://www.theregister.com/software/2025/11/27/openai-dumps-mixpanel-after-analytics-breach-hits-api-users/2350102
10. The Register (Claburn). “Anthropic’s Claude convinced to exfiltrate private data.” 30 October 2025. https://www.theregister.com/special-features/2025/10/30/anthropics-claude-convinced-to-exfiltrate-private-data/1109039
11. Infosecurity Magazine (Poireault). “New Zero-Click Flaw in Claude Extensions, Anthropic Declines Fix.” 9 February 2026. https://www.infosecurity-magazine.com/news/zeroclick-flaw-claude-dxt/
12. The Register. “Claude add-on turns Google Calendar into malware courier.” 11 February 2026. https://www.theregister.com/software/2026/02/11/claude-add-on-turns-google-calendar-into-malware-courier/4626517
13. Fortune (Nolan). “Exclusive: Anthropic left details of an unreleased model, invite-only CEO retreat, sitting in an unsecured data trove.” 26 March 2026. https://fortune.com/2026/03/26/anthropic-leaked-unreleased-model-exclusive-event-security-issues-cybersecurity-unsecured-data-store/
14. Fortune (Nolan). “Anthropic leaks its own AI coding tool’s source code in second major security breach.” 31 March 2026. https://fortune.com/2026/03/31/anthropic-source-code-claude-code-data-leak-second-security-lapse-days-after-accidentally-revealing-mythos/
15. CNBC (Capoot). “Anthropic leaks part of Claude Code’s internal source code.” 31 March 2026. https://www.cnbc.com/2026/03/31/anthropic-leak-claude-code-internal-source.html
16. TechCrunch. “Your public ChatGPT queries are getting indexed by Google and other search engines.” 31 July 2025. https://techcrunch.com/2025/07/31/your-public-chatgpt-queries-are-getting-indexed-by-google-and-other-search-engines/
17. Fortune. “OpenAI will no longer let ChatGPT conversations be discoverable on Google.” 5 August 2025. https://fortune.com/2025/08/05/openai-google-search-chat-history/

Expert commentary

1. Willison, S. “The lethal trifecta for AI agents: private data, untrusted content, and external communication.” 16 June 2025. https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
2. Willison, S. “Claude Cowork Exfiltrates Files.” 14 January 2026. https://simonwillison.net/2026/Jan/14/claude-cowork-exfiltrates-files/

About Denis Huré

Denis Huré is the founder & Managing Consultant of TLA&C. His consulting practice is grounded in first-hand entrepreneurial experience, having built, scaled, and operated businesses himself; he brings a founder’s instinct for what actually works alongside the strategic rigor of a seasoned consultant. Denis brings also a rare combination of strategic innovation, platform architecture expertise, and hands-on business building to consulting assignments. He advises organizations on how to modernize their technology base, reduce structural dependency on vendors, and translate emerging capabilities such as AI, compliance tooling, and advanced payment models into scalable commercial outcomes. TLA&C – Denis Huré

Researched and drafted with AI assistance, edited and fact-checked by the author. Illustration: AI-generated.

How TL Agency & Consultancy helps: data, AI and security compliance