  Effective representation of data is crucial in various machine learning
tasks, as it captures the underlying structure and context of the data.
Embeddings have emerged as a powerful technique for data representation, but
evaluating their quality and capacity to preserve structural and contextual
information remains a challenge. In this paper, we address this need by
proposing a method to measure the \textit{representation capacity} of
embeddings. The motivation behind this work stems from the importance of
understanding the strengths and limitations of embeddings, enabling researchers
and practitioners to make informed decisions in selecting appropriate embedding
models for their specific applications. By combining extrinsic evaluation
methods, such as classification and clustering, with t-SNE-based neighborhood
analysis, such as neighborhood agreement and trustworthiness, we provide a
comprehensive assessment of the representation capacity. Additionally, the use
of optimization techniques (bayesian optimization) for weight optimization (for
classification, clustering, neighborhood agreement, and trustworthiness)
ensures an objective and data-driven approach in selecting the optimal
combination of metrics. The proposed method not only contributes to advancing
the field of embedding evaluation but also empowers researchers and
practitioners with a quantitative measure to assess the effectiveness of
embeddings in capturing structural and contextual information. For the
evaluation, we use $3$ real-world biological sequence (proteins and nucleotide)
datasets and performed representation capacity analysis of $4$ embedding
methods from the literature, namely Spike2Vec, Spaced $k$-mers, PWM2Vec, and
AutoEncoder.
