  Unsupervised embeddings are fundamental to numerous machine learning
applications, yet their evaluation remains a challenging task. Traditional
assessment methods often rely on extrinsic variables, such as performance in
downstream tasks, which can introduce confounding factors and mask the true
quality of embeddings. This paper introduces the Intrinsic Distance
Preservation Evaluation (IDPE) method, a novel approach for assessing embedding
quality based on the preservation of Mahalanobis distances between data points
in the original and embedded spaces. We demonstrate the limitations of
extrinsic evaluation methods through a simple example, highlighting how they
can lead to misleading conclusions about embedding quality. IDPE addresses
these issues by providing a task-independent measure of how well embeddings
preserve the intrinsic structure of the original data. Our method leverages
efficient similarity search techniques to make it applicable to large-scale
datasets. We compare IDPE with established intrinsic metrics like
trustworthiness and continuity, as well as extrinsic metrics such as Average
Rank and Mean Reciprocal Rank. Our results show that IDPE offers a more
comprehensive and reliable assessment of embedding quality across various
scenarios. We evaluate PCA and t-SNE embeddings using IDPE, revealing insights
into their performance that are not captured by traditional metrics. This work
contributes to the field by providing a robust, efficient, and interpretable
method for embedding evaluation. IDPE's focus on intrinsic properties offers a
valuable tool for researchers and practitioners seeking to develop and assess
high-quality embeddings for diverse machine learning applications.
