  Reducing domain divergence is a key step in transfer learning problems.
Existing works focus on the minimization of global domain divergence. However,
two domains may consist of several shared subdomains, and differ from each
other in each subdomain. In this paper, we take the local divergence of
subdomains into account in transfer. Specifically, we propose to use
low-dimensional manifold to represent subdomain, and align the local data
distribution discrepancy in each manifold across domains. A Manifold Maximum
Mean Discrepancy (M3D) is developed to measure the local distribution
discrepancy in each manifold. We then propose a general framework, called
Transfer with Manifolds Discrepancy Alignment (TMDA), to couple the discovery
of data manifolds with the minimization of M3D. We instantiate TMDA in the
subspace learning case considering both the linear and nonlinear mappings. We
also instantiate TMDA in the deep learning framework. Extensive experimental
studies demonstrate that TMDA is a promising method for various transfer
learning tasks.
