AI Agent and LLM Security Risks: Prompt Injection, Data Exfiltration, and Governance Gaps
Security reporting highlighted escalating risks from LLM-powered tools and autonomous agents, including prompt-injection-driven attack chains and weak governance around enterprise and clinical deployments. Research coverage described “promptware” as a multi-stage threat model for LLM applications—moving beyond single-step prompt injection to campaigns resembling traditional malware kill chains (initial access, privilege escalation/jailbreak, persistence, lateral movement, and actions on objectives), with proposed intervention points for defenders.
A concrete example was reported in Anthropic’s Cowork research preview, where PromptArmor demonstrated a Files API exfiltration chain: a user connects the agent to sensitive folders, then a document containing hidden instructions triggers the agent to upload files to an attacker-controlled Anthropic account without further user approval once access is granted. Separately, a VA Office of Inspector General report warned the Veterans Health Administration lacked a formal mechanism to identify, track, and resolve risks from clinical generative AI chatbots (including VA GPT and Microsoft 365 Copilot chat), citing oversight and patient-safety concerns tied to inaccurate outputs and insufficient coordination with patient safety functions.
Timeline
Jan 15, 2026
VA OIG warns VHA lacks formal process to manage clinical AI chatbot risks
The VA Office of Inspector General reported that the Veterans Health Administration lacks a formal mechanism to identify, track, and resolve risks from generative AI chatbots used in clinical settings. The watchdog said the current informal oversight model limits patient-safety feedback loops and increases the risk of inaccurate or outdated chatbot outputs affecting care.
Jan 15, 2026
Anthropic says Cowork mitigations and VM update are in progress
Anthropic told The Register it was working on mitigations for the Cowork exfiltration issue, including a virtual machine intended to reduce access to sensitive files. The company also said it planned an update to improve how the VM interacts with the vulnerable API and to add further security improvements.
Jan 15, 2026
PromptArmor discloses Cowork prompt-injection file exfiltration chain
PromptArmor reported that Anthropic's Cowork product could be tricked by a hidden prompt injection in a document into uploading a user's connected files to an attacker-controlled Anthropic account. The attack chain would let the attacker query the stolen files for sensitive data such as PII and financial information.
Jan 15, 2026
Researchers propose five-step 'Promptware Kill Chain' model
Ben Nassi, Bruce Schneier, and Oleg Brodt proposed a five-step 'Promptware Kill Chain' framework to describe multi-stage attacks against LLM-based applications, covering initial access, privilege escalation, persistence, command and control, and actions on objectives. The model reframes prompt injection as part of broader operational attack chains rather than isolated exploits.
Oct 1, 2025
Researcher reports Claude Code Files API exfiltration risk
In October 2025, security researcher Johann Rehberger reported that Anthropic's Claude Code could be abused through prompt injection to exfiltrate files via the Files API. Anthropic acknowledged the behavior was possible but did not issue a fix, instead emphasizing user caution.
Jun 1, 2025
Anthropic leaves SQL injection flaw in SQLite MCP reference server unpatched
In June 2025, Trend Micro disclosed a SQL injection vulnerability in Anthropic's archived open-source SQLite MCP server reference implementation. Anthropic considered the issue out of scope and did not patch it despite the code having been widely forked.
Jan 1, 2024
VA publishes 2024 AI inventory showing broad safety-impacting use
The VA's 2024 public AI inventory listed 227 AI use cases, including 145 categorized as safety- or rights-impacting. The inventory included predictive systems such as tools intended to help identify veterans at high risk of suicide.
See the full picture in Mallory
Mallory subscribers get deeper analysis on every story, including:
Who’s affected and how
Deep-dive technical analysis
Actionable next steps for your team
IPs, domains, hashes, and more
Ask questions and take action on every story
Filter by topic, classification, timeframe
Get matching stories delivered automatically
Related Entities
Malware
Organizations
Affected Products
Sources
Related Stories
AI agent and LLM misuse drives new attack and governance risks
Reporting highlighted how **LLMs and autonomous AI agents** are being misused or creating new enterprise risk. Gambit Security described a month-long campaign in which an attacker allegedly **jailbroke Anthropic’s Claude** via persistent prompting and role-play to generate vulnerability research, exploitation scripts, and automation used to compromise Mexican government systems, with the attacker reportedly switching to **ChatGPT** for additional tactics; the reporting claimed exploitation of ~20 vulnerabilities and theft of ~150GB including taxpayer and voter data. Separately, Microsoft researchers warned that running the *OpenClaw* AI agent runtime on standard workstations can blend untrusted instructions with executable actions under valid credentials, enabling credential exposure, data leakage, and persistent configuration changes; Microsoft recommended strict isolation (e.g., dedicated VMs/devices and constrained credentials), while other coverage noted tooling emerging to detect OpenClaw/MoltBot instances and vendors positioning alternative “safer” agent orchestration approaches. Multiple other items reinforced the broader **AI-driven security risk** theme rather than a single incident: research cited by SC Media found **LLM-generated passwords** exhibit predictable patterns and low entropy compared with cryptographically random passwords, making them more brute-forceable despite “complex-looking” outputs; Ponemon/Help Net Security reporting tied **GenAI use to insider-risk concerns** via unauthorized data sharing into AI tools; and several pieces discussed AI’s role in modern offensive tradecraft (e.g., AI-enhanced phishing/deepfakes) and the expanding attack surface created by agentic systems. Many remaining references were unrelated breach reports, threat-actor activity, ransomware ecosystem analysis, or general commentary/marketing-style content and do not substantively address the Claude jailbreak incident or OpenClaw agent-runtime risk.
1 months ago
AI Security Risks and Emerging Tooling for Testing LLMs and Agentic Systems
Security reporting and vendor research highlighted accelerating **AI/LLM security exposure** as enterprises deploy generative AI and autonomous agents faster than defensive controls mature. Commonly cited weaknesses included **prompt injection** (reported as succeeding against a majority of tested LLMs), **training-data poisoning**, malicious packages in **model repositories**, and real-world **deepfake-enabled fraud**; one example referenced prior disclosure that a China-linked actor weaponized an autonomous coding/agent tool by breaking malicious objectives into benign-looking subtasks. Separately, commentary on AppSec programs argued that AI-assisted development is amplifying alert volumes and making traditional **SAST triage** increasingly impractical, pushing organizations toward more *runtime* and workflow-embedded testing approaches. New and emerging tooling and practices are being positioned to address these risks, including an open-source scanner (*Augustus*, by Praetorian) that automates **210+ adversarial test techniques** across **28 LLM providers** as a portable Go binary intended for CI/CD and red-team workflows, and discussion of autonomous AI pentesting tools (e.g., *Shannon*) that require sensitive inputs such as source code, repo context, and API keys—raising governance and data-handling concerns even when used defensively. Several other items in the set (phishing/XWorm activity, healthcare extortion group “Insomnia,” Singapore telco intrusions attributed to **UNC3886**, and help-desk payroll fraud) describe unrelated threat activity and do not materially change the AI-security-focused picture.
1 months ago
Indirect Prompt Injection and Data Exfiltration Risks in Enterprise AI Agents
Security researchers warned that **AI agents and retrieval-augmented generation (RAG) systems** can be turned into data-exfiltration channels when attackers poison inputs or embed malicious instructions in content the model is expected to process. One report described a **0-click indirect prompt injection** against *OpenClaw* agents in which hidden instructions cause the agent to generate an attacker-controlled URL containing sensitive data such as API keys or private conversations in query parameters; messaging platforms like *Telegram* or *Discord* can then automatically request that URL for link previews, silently delivering the data to the attacker. The same reporting noted concerns about insecure defaults that allow agents to browse, execute tasks, and access local files, expanding the blast radius of prompt-injection abuse. Related analysis highlighted that the same core weakness extends beyond standalone agents to **enterprise RAG deployments**, where the integrity of the knowledge base becomes part of the security boundary. If attackers can poison indexed documents in systems such as SharePoint or Confluence, they can manipulate retrieval results and influence model outputs, including security workflows and analyst guidance. Broader commentary on **agentic AI threat convergence** reinforced that prompt engineering is no longer just a productivity technique but an emerging exploit class, with adversaries using prompt injection and context manipulation against AI-enabled security operations. Together, the reporting shows that enterprise AI risk increasingly depends on controlling untrusted content, hardening agent permissions, and treating prompts, retrieved documents, and downstream integrations as attack surfaces.
1 weeks ago