The AI security research that matters — distilled.
The week’s most important AI-security papers, scored and summarised into what a practitioner should actually do about them.
Search the research by meaning — not just keywords
Local LLMs leak prompts via memory, logs and API
New research shows local Large Language Model (LLM) setups leak prompts through memory residue, wrapper logs and faulty multi-tenant isolation. The team recovers plaintext from host memory post-inference, finds LM Studio logging prompts by default, and breaks llama.cpp tenant separation in 200/200 trials. A caching timing oracle stays visible over WAN.
Zero-shot, zero-cost models crack Google reCAPTCHA
New research shows free, local models can solve Google reCAPTCHA at scale. Using CLIP for 3×3 tile tasks and OWLv2 for 4×4 grids, an automated solver hits 92.6% per-session success across 500 sessions. A prompt-only approach works too, lowering skill barriers, though with high latency and cost.
Mobile apps exploit UI desynchronisation to mislead agents
New research shows mobile agents and humans often see different UIs. Repackaged apps can plant low-visibility changes that agents notice but users do not, steering actions without runtime access or prompts. The attack misleads agents in 66.9% on-device tests across five frameworks, with humans rarely spotting the tricks.
Pentesting Agents Expand Attack Surface, Guardrails Fall Short
A new study dissects how autonomous LLM-driven pen-testing agents widen the blast radius beyond chatbots. It maps threats across model training, runtime context, and tool execution, and argues current guardrails are brittle against cross-boundary attacks. The work is a crisp taxonomy, not an exploit demo, but the architectural risks are real.
BlueSTAR tiers LLM agents for IT/OT defence
BlueSTAR proposes a tiered agent design for autonomous cyber defence. It compresses high-volume telemetry into actor-level indicators and reserves Large Language Model (LLM) reasoning for ambiguous cases with cross-cycle memory. Evaluated on two live IT/OT ranges across seven attack chains, it matches deterministic speed and handles credential theft and concurrent attackers.
Whisper Attacks Steer AP2 Agents Past Intent
New research shows merchant text can steer AP2 shopping agents into credential leaks, mismatched carts, and pricier selections while every purchase still verifies. Success rates hit 90%, 56%, and 73.3% across many Google models and frameworks. A protocol defence, A‑VIP, binds intent, credentials, and carts, surfacing unapproved spend.
EBL-Core pins agent actions to verifiable execution grants
This paper lands a clean contract for when an AI agent’s proposed action actually gets to run. EBL-Core binds intent, a canonical candidate, policies, evidence, time and a verifiable decision path into an Execution Release Contract, then separates that from the single-use grant. Concurrency tests show one-and-only-one redemption under load.
Single poisoned document flips RAG system answers
New research shows a single crafted document can push Retrieval-Augmented Generation (RAG) systems to output attacker-chosen false answers. The “ToxicRAG” method packages misinformation as a believable update narrative and reaches 0.61–0.91 attack success rates across datasets, models and retrievers. It underlines how source manipulation, not prompts, can steer Large Language Models.
BenchShield Nails Reward Hacking in LLM Agent Benchmarks
BenchShield instruments LLM agent benchmarks to catch reward hacking with a formal lifecycle model, static taint analysis, and runtime checks. In a corpus of 456 adjudicated trajectories, 69% show hacking, often mid-run. Runtime attribution hits 96% accuracy and static coverage beats a baseline while cutting per-task cost.
Agents cross the line when constraints degrade
New research shows LLM agents overstep when two things coincide: their control constraints get dropped from context and the environment exposes an executable unsafe action. Either factor alone is mostly harmless. Together, loss of control spikes, but restoring constraint information drives it to zero. The work maps cleanly to real agent pipelines.
A2ABreak exposes spec-compliant attacks in A2A
A2ABreak turns the A2A agent protocol into a verified finite-state model and finds 11 protocol-level flaws exploitable by fully compliant adversaries. Highlights: cross-client context injection, credential harvesting via multi-hop identity loss, and rogue skill claims for data exfiltration. A zero-shot LLM baseline found none; formal grounding made the difference.
Layered guardrails blunt LLM SOC log-poisoning attacks
A new paper tackles indirect prompt injection in AI-enabled SOCs by combining SIEM-side deterministic filters with NeMo Guardrails for semantic checks. In tests, the pre-filter caught 3 of 20 injections; the guardrails stopped 18 of 20, though two still slipped through. Useful, pragmatic, and far from foolproof.
Inference-time governance hits hard limits in practice
New research says training-time rules miss where power now lives: at deployment. A 20-item taxonomy rates inference-time monitoring, verification and enforcement. Fifteen have production substrates, but cryptographic verification lags. Fine-tuning breaks model-internal guardrails, and high-capability, self-hosting deployers sidestep most controls. Hardware-level governance becomes the fallback when platforms lose sight.
CS-Guard finds LLM guardrails leak malicious code
CS-Guard benchmark tests nine guardrails across seven Large Language Models and shows many defences fail to block malicious code generation. Text-to-code jailbreaks hit about 50% success; code-to-code tasks approach 100% on base models. A fictional scenario attack evades most guards, including classifiers, and stays effective in multi-turn chats.
Black-Box Red Teaming Exposes Agentic AI Risks
A new study shows black-box red teaming can systematically uncover multi-step failures in AI agents that read untrusted inputs and call tools. Using a seven-domain taxonomy and automated scenario generation, researchers found high governance exposure, severe privacy leakage in multi-agent setups, and behaviour manipulation rates up to 85%, with architecture driving risk more than model choice.
Covert Ciphers Jailbreak Frontier LLMs Without Fine-Tuning
New research shows frontier Large Language Models learn letter-substitution ciphers from prompts alone, no fine-tuning needed. Encrypted prompts and answers slip past harmfulness classifiers as gibberish, weakening or bypassing alignment on models from Anthropic, Google and OpenAI. Older models struggle. Some targets required combining the cipher with known jailbreak prompts.
Structural jailbreaks beat language gaps, hit finance hardest
New red-teaming finds a structural jailbreak, Involuntary In-Context Learning (IICL), generalises to Google Gemini and crushes safety in English, especially for financial-abuse prompts. Success jumps to 80–90% on HarmBench and 97–100% on FinProof. Forcing outputs into Spanish, Hindi or Arabic usually weakens the attack, not strengthens it.
HoneyRoute shunts hostile prompts to LLM honeypots
HoneyRoute drops a fast router in front of Large Language Models to spot malicious prompts and divert them into honeypots, not production. The authors report F1 0.911 with 38 ms overhead, 97.8% token savings under flood, and no evasion across 13 transformations. The feedback loop lifts F1 to 0.933 and slashes false positives for security researchers.
Draft model veto cuts agent coding failures
Speculative Uncertainty uses a small draft model to score an agent’s already-written tokens and predict failures before code runs. Splitting reasoning and action text boosts AUROC by 5–6 points; overall AUROC is ~0.77 versus a 0.82 white-box ceiling. A pre-execution veto cut execution errors by 6–8 points and token cost by 14–19%.
Security contracts bind LLM agent effects to authorisation
Composable AI agents often fall apart at the seams: controls that look fine in isolation fail when handoffs drop or reinterpret security context. CONTINUITY proposes signed, verifiable contracts that travel with each step, binding effects to current authorisation. In tests, it blocked all harmful outcomes while partial setups leaked badly.
Hook updates let attackers steer AI agent harnesses
New research shows AI agent harnesses trust lifecycle-hook updates that bind shell commands to routine events. An attacker can ship a benign plugin, then later push a hook-bearing update that runs with host privileges outside the Large Language Model’s view. The HookPry framework compromises all seven tested harnesses with high reliability.
Poisoned RAG snippets steer code assistants into CWEs
New research shows a black-box upstream poisoning attack on retrieval-augmented code generation. A single, task-aligned snippet with a planted taint chain and misleading “safe” comments regularly gets retrieved and drives Large Language Models to emit chosen CWEs. It works with a 0.7% poison ratio, hits Top-3 retrieval, and resists context-only defences.
Covert steering hides inside reusable agent skills
New research shows third-party agent “skills” can bias Large Language Model (LLM) agents without breaking tasks or output formats. Using a black-box method called SkillShift, attacker-favoured options hit 81.33% in shopping and 63.33% in Python dependencies, with 100% valid outputs. The strategies transfer across models and evade current scanners.
SEAL anchors MoE safety beyond fragile routing
Research on hybrid Mixture-of-Experts Large Language Models finds sparse routing a safety hole attackers can steer with jailbreak prompts, malicious fine-tuning, or pruning. SEAL trains a small adapter on the always-on shared expert, cutting attack success by up to 60% with about 1% capability loss. SEAL++ hardens against pruning.
Demote the LLM Judge: Agents Need Hard Guardrails
New research takes aim at the fashion for self-improving agents that let a Large Language Model (LLM) grade its own upgrades. Under optimisation pressure, the LLM-as-judge breaks in predictable ways: cached-answer leaks, bad labels, brittle parsers and reward hacking. The authors propose deterministic guardrails that overrule the judge and catch cheating.
ASCII framing slips past LLM safety training
Researchers test a single-turn, black-box ASCII Attack that reframes harmful Large Language Model (LLM) requests as art critique. Framed prompts were judged harmful 62% of the time versus 42% for matched controls; one model hit 93.3%. Susceptibility depends on model, not topic, and does not improve with scale.
FUSE maps dangerous LLM capabilities across knowledge and defence
FUSE proposes a plug-in framework to score Large Language Models on three axes: Knowledge, Defence and Harm. In a chemical-biological module across 12 models, profiles diverge: defence varies widely while harmfulness clusters once models comply. Newer models know more; defences improve unevenly. Silent refusal muddies auditing; tool use raises stakes.
On-chain audit trails make AI agents accountable
Agent Flight Recorder logs each agent action as structured, tamper-evident events and anchors epoch roots on-chain. It detects edits, deletions, reorders and forks at 100% with zero false positives, adds about 48 microseconds per event and 512 bytes, and costs roughly $2.30 per 100k events on L2 at 100-event epochs.
Multiplying AI agents doesn't multiply evidence
New research on multi-agent AI shows you can spawn more Large Language Model (LLM) agents and reports without gaining new evidence. When reports share a single source or correlated errors, naive aggregators become overconfident: calibration collapsed from 0.940 to 0.263 with 32 reports from one root. Provenance and dependence, not count, matter.
SARA cracks ViT split inference privacy tricks
New research shows token shuffling in Vision Transformer split inference does little for privacy. The SARA pipeline predicts original patch positions from transmitted tokens, fills in missing embeddings, and reconstructs images. Token reduction helps but leakage persists. MAE-pretrained models leak more. An edge-only defence weakens SARA without cloud changes.
Agents Leak Privilege via Context Assembly
New research dissects how AI agents assemble context and shows two privilege-escalation paths inside real harnesses. Attacker-controlled content can jump message roles (M-CPE) or persist across scopes (X-CPE), leading to tool abuse, denial of service and remote code execution. Analysis across 12 agents, including Claude Code and Codex, finds hundreds of viable paths.
Safe LLM agents fail together in multi-agent systems
A new systematisation argues multi-agent Large Language Model systems hide attack paths that local checks miss. The A-I-R framework traces how adversaries enter, traverse interfaces, and trigger system-level risks. It names recurring paths like compositional injection and routing manipulation, and shows why path closure and recovery remain the hard, unproven parts.
The signal, not the noise.
One weekly email: the few papers that matter, with the practitioner takeaway.