  This paper introduces a multi-round interaction problem with privacy
constraints between two agents that observe correlated data. The agents
alternately share data with one another for a total of K rounds such that each
agent initiates sharing over K/2 rounds. The interactions are modeled as a
collection of K random mechanisms (mappings), one for each round. The goal is
to jointly design the K private mechanisms to determine the set of all
achievable distortion-leakage pairs at each agent. Arguing that a mutual
information-based leakage metric can be appropriate for streaming data
settings, this paper: (i) determines the set of all achievable distortion-
leakage tuples ; (ii) shows that the K mechanisms allow for precisely composing
the total privacy budget over K rounds without loss; and (ii) develops
conditions under which interaction reduces the net leakage at both agents and
illustrates it for a specific class of sources. The paper then focuses on
log-loss distortion to better understand the effect on leakage of using a
commonly used utility metric in learning theory. The resulting interaction
problem leads to a non-convex sum-leakage-distortion optimization problem that
can be viewed as an interactive version of the information bottleneck problem.
A new merge-and-search algorithm that extends the classical agglomerative
information bottleneck algorithm to the interactive setting is introduced to
determine a provable locally optimal solution. Finally, the benefit of
interaction under log-loss is illustrated for specific source classes and the
optimality of one-shot is proved for Gaussian sources under both mean-square
and log-loss distortions constraints.
