This paper proposes a method to improve the robustness of large language model agents against indirect prompt injection attacks by using a unified framework that combines red-teaming and adversarial training. Practitioners might care because it could help prevent agents from being hijacked by attackers and ensure they can complete their intended tasks.
Firehose
Filtered to tagged “Prompt injection” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 63open-weight models 19agentic coding 17continual learning 16AI 15AI agents 11reinforcement learning 9datasette 8AI safety 7cybersecurity 7Databricks 7natural language processing 7AI Gateway 6large language models 6Reinforcement learning 6vision-language models 6computer vision 5Diffusion models 5language models 5machine learning 5open-source 5Agentic AI 4AI ethics 4benchmarks 4education 4productivity 4recursive self-improvement 4Recursive self-improvement 4space 4tech 4
This episode features Zico Kolter and Matt Fredrikson from Grey Swan discussing AI security challenges, particularly around adversarial attacks and indirect prompt injection in large language models and AI agents. They explain their approac…
AI in the AM: 99% off search, GPT-5.5 is "clean", model welfare analysis, & efficient analog compute
This AI in the AM episode features four guests discussing recent advancements and challenges in AI. Anna Patterson introduces Ceramic AI's low-cost enterprise search for LLMs, designed to enhance fact-checking and combine public/private dat…