  Cosine similarity is widely used to measure the similarity between two
embeddings, while interpretations based on angle and correlation coefficient
are common. In this study, we focus on the interpretable axes of embeddings
transformed by Independent Component Analysis (ICA), and propose a novel
interpretation of cosine similarity as the sum of semantic similarities over
axes. To investigate this, we first show experimentally that unnormalized
embeddings contain norm-derived artifacts. We then demonstrate that normalized
ICA-transformed embeddings exhibit sparsity, with a few large values in each
axis and across embeddings, thereby enhancing interpretability by delineating
clear semantic contributions. Finally, to validate our interpretation, we
perform retrieval experiments using ideal embeddings with and without specific
semantic components.
