  The growing use of large language model (LLM)-based conversational agents to
manage sensitive user data raises significant privacy concerns. While these
agents excel at understanding and acting on context, this capability can be
exploited by malicious actors. We introduce a novel threat model where
adversarial third-party apps manipulate the context of interaction to trick
LLM-based agents into revealing private information not relevant to the task at
hand.
  Grounded in the framework of contextual integrity, we introduce AirGapAgent,
a privacy-conscious agent designed to prevent unintended data leakage by
restricting the agent's access to only the data necessary for a specific task.
Extensive experiments using Gemini, GPT, and Mistral models as agents validate
our approach's effectiveness in mitigating this form of context hijacking while
maintaining core agent functionality. For example, we show that a single-query
context hijacking attack on a Gemini Ultra agent reduces its ability to protect
user data from 94% to 45%, while an AirGapAgent achieves 97% protection,
rendering the same attack ineffective.
