  We develop a family of techniques to align word embeddings which are derived
from different source datasets or created using different mechanisms (e.g.,
GloVe or word2vec). Our methods are simple and have a closed form to optimally
rotate, translate, and scale to minimize root mean squared errors or maximize
the average cosine similarity between two embeddings of the same vocabulary
into the same dimensional space. Our methods extend approaches known as
Absolute Orientation, which are popular for aligning objects in
three-dimensions, and generalize an approach by Smith etal (ICLR 2017). We
prove new results for optimal scaling and for maximizing cosine similarity.
Then we demonstrate how to evaluate the similarity of embeddings from different
sources or mechanisms, and that certain properties like synonyms and analogies
are preserved across the embeddings and can be enhanced by simply aligning and
averaging ensembles of embeddings.
