  In this paper, we propose a new kernel-based co-occurrence measure that can
be applied to sparse linguistic expressions (e.g., sentences) with a very short
learning time, as an alternative to pointwise mutual information (PMI). As well
as deriving PMI from mutual information, we derive this new measure from the
Hilbert--Schmidt independence criterion (HSIC); thus, we call the new measure
the pointwise HSIC (PHSIC). PHSIC can be interpreted as a smoothed variant of
PMI that allows various similarity metrics (e.g., sentence embeddings) to be
plugged in as kernels. Moreover, PHSIC can be estimated by simple and fast
(linear in the size of the data) matrix calculations regardless of whether we
use linear or nonlinear kernels. Empirically, in a dialogue response selection
task, PHSIC is learned thousands of times faster than an RNN-based PMI while
outperforming PMI in accuracy. In addition, we also demonstrate that PHSIC is
beneficial as a criterion of a data selection task for machine translation
owing to its ability to give high (low) scores to a consistent (inconsistent)
pair with other pairs.
