  We discuss the feasibility of the following learning problem: given unmatched
samples from two domains and nothing else, learn a mapping between the two,
which preserves semantics. Due to the lack of paired samples and without any
definition of the semantic information, the problem might seem ill-posed.
Specifically, in typical cases, it seems possible to build infinitely many
alternative mappings from every target mapping. This apparent ambiguity stands
in sharp contrast to the recent empirical success in solving this problem.
  We identify the abstract notion of aligning two domains in a semantic way
with concrete terms of minimal relative complexity. A theoretical framework for
measuring the complexity of compositions of functions is developed in order to
show that it is reasonable to expect the minimal complexity mapping to be
unique. The measured complexity used is directly related to the depth of the
neural networks being learned and a semantically aligned mapping could then be
captured simply by learning using architectures that are not much bigger than
the minimal architecture.
  Various predictions are made based on the hypothesis that semantic alignment
can be captured by the minimal mapping. These are verified extensively. In
addition, a new mapping algorithm is proposed and shown to lead to better
mapping results.
