  Context: As mobile applications (Apps) widely spread over our society and
life, various personal information is constantly demanded by Apps in exchange
for more intelligent and customized functionality. An increasing number of
users are voicing their privacy concerns through app reviews on App stores.
  Objective: The main challenge of effectively mining privacy concerns from
user reviews lies in the fact that reviews expressing privacy concerns are
overridden by a large number of reviews expressing more generic themes and
noisy content. In this work, we propose a novel automated approach to overcome
that challenge.
  Method: Our approach first employs information retrieval and document
embeddings to unsupervisedly extract candidate privacy reviews that are further
labeled to prepare the annotation dataset. Then, supervised classifiers are
trained to automatically identify privacy reviews. Finally, we design an
interpretable topic mining algorithm to detect privacy concern topics contained
in the privacy reviews.
  Results: Experimental results show that the best performed document embedding
achieves an average precision of 96.80% in the top 100 retrieved candidate
privacy reviews. All of the trained privacy review classifiers can achieve an
F1 value of more than 91%, outperforming the recent keywords matching baseline
with the maximum F1 margin being 7.5%. For detecting privacy concern topics
from privacy reviews, our proposed algorithm achieves both better topic
coherence and diversity than three strong topic modeling baselines including
LDA.
  Conclusion: Empirical evaluation results demonstrate the effectiveness of our
approach in identifying privacy reviews and detecting user privacy concerns
expressed in App reviews.
