  Recently many efforts have been made to incorporate persistence diagrams, one
of the major tools in topological data analysis (TDA), into machine learning
pipelines. To better understand the power and limitation of persistence
diagrams, we carry out a range of experiments on both graph data and shape
data, aiming to decouple and inspect the effects of different factors involved.
To this end, we also propose the so-called \emph{permutation test} for
persistence diagrams to delineate critical values and pairings of critical
values. For graph classification tasks, we note that while persistence pairing
yields consistent improvement over various benchmark datasets, it appears that
for various filtration functions tested, most discriminative power comes from
critical values. For shape segmentation and classification, however, we note
that persistence pairing shows significant power on most of the benchmark
datasets, and improves over both summaries based on merely critical values, and
those based on permutation tests. Our results help provide insights on when
persistence diagram based summaries could be more suitable.
