Interpreting Jailbreaks and Prompt Injections with Attribution Graphs - https://labs.zenity.io/p/interpreting-jailbreaks-and-prompt-injections-with-attribution-graphs by @zenitysec

Today’s agent security is strong at the edges: we monitor inputs/outputs, trace and permission tool calls, track taint, rate-limit, and log everything. We have a very complex agent system that we break down into components and secure each of them.

Yet the LLM at the heart of the agent remains a box that we never open. This is akin to a medicine that treats symptoms without understanding the underlying mechanism that causes them.

In parallel, the field of mechanistic interpretability (interpretability that looks at internal states) for LLMs has been showing increasingly fascinating findings, allowing us, for the first time, to glimpse inside the LLM and find interpretable features and the circuits that use them to build the model response to a given input.

We’ve decided these 2 should be combined and have embarked on a journey to research LLM internals to better understand and improve security of AI agents.
This will be the first in a series of posts describing this journey.