  Advanced AI assistants combine frontier LLMs and tool access to autonomously
perform complex tasks on behalf of users. While the helpfulness of such
assistants can increase dramatically with access to user information including
emails and documents, this raises privacy concerns about assistants sharing
inappropriate information with third parties without user supervision. To steer
information-sharing assistants to behave in accordance with privacy
expectations, we propose to operationalize $\textit{contextual integrity}$
(CI), a framework that equates privacy with the appropriate flow of information
in a given context. In particular, we design and evaluate a number of
strategies to steer assistants' information-sharing actions to be CI compliant.
Our evaluation is based on a novel form filling benchmark composed of synthetic
data and human annotations, and it reveals that prompting frontier LLMs to
perform CI-based reasoning yields strong results.
