  Asymmetric data naturally exist in real life, such as directed graphs.
Different from the common kernel methods requiring Mercer kernels, this paper
tackles the asymmetric kernel-based learning problem. We describe a nonlinear
extension of the matrix Singular Value Decomposition through asymmetric
kernels, namely KSVD. First, we construct two nonlinear feature mappings w.r.t.
rows and columns of the given data matrix. The proposed optimization problem
maximizes the variance of each mapping projected onto the subspace spanned by
the other, subject to a mutual orthogonality constraint. Through Lagrangian
duality, we show that it can be solved by the left and right singular vectors
in the feature space induced by the asymmetric kernel. Moreover, we start from
the integral equations with a pair of adjoint eigenfunctions corresponding to
the singular vectors on an asymmetrical kernel, and extend the Nystr\"om method
to asymmetric cases through the finite sample approximation, which can be
applied to speedup the training in KSVD. Experiments show that asymmetric KSVD
learns features outperforming Mercer-kernel based methods that resort to
symmetrization, and also verify the effectiveness of the asymmetric Nystr\"om
method.
