  Kernel methods have been among the most popular techniques in machine
learning, where learning tasks are solved using the property of reproducing
kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis
framework with reproducing kernel Hilbert $C^*$-module (RKHM), which is another
generalization of RKHS than vector-valued RKHS (vv-RKHS). Analysis with RKHMs
enables us to deal with structures among variables more explicitly than
vv-RKHS. We show the theoretical validity for the construction of orthonormal
systems in Hilbert $C^*$-modules, and derive concrete procedures for
orthonormalization in RKHMs with those theoretical properties in numerical
computations. Moreover, we apply those to generalize with RKHM kernel principal
component analysis and the analysis of dynamical systems with Perron-Frobenius
operators. The empirical performance of our methods is also investigated by
using synthetic and real-world data.
