  Large language models (LLMs) have had a profound impact on numerous aspects
of daily life including natural language processing, content generation,
research methodologies and so on. However, one crucial issue concerning the
inference results of large language models is security and privacy. In many
scenarios, the results generated by LLMs could possibly leak many confidential
or copyright information. A recent beautiful and breakthrough work [Vyas,
Kakade and Barak 2023] focus on such privacy issue of the LLMs from theoretical
perspective. It is well-known that computing the attention matrix is one of the
major task during the LLMs computation. Thus, how to give a provable privately
guarantees of computing the attention matrix is an important research
direction.
  Previous work [Alman and Song 2023, Brand, Song and Zhou 2023] have proposed
provable tight result for fast computation of attention without considering
privacy concerns. One natural mathematical formulation to quantity the privacy
in theoretical computer science graduate school textbook is differential
privacy. Inspired by [Vyas, Kakade and Barak 2023], in this work, we provide a
provable result for showing how to differentially private approximate the
attention matrix.
  From technique perspective, our result replies on a pioneering work in the
area of differential privacy by [Alabi, Kothari, Tankala, Venkat and Zhang
2022].
