  Federated Learning (FL) is a privacy-preserving distributed machine learning
paradigm. Nonetheless, the substantial distribution shifts among clients pose a
considerable challenge to the performance of current FL algorithms. To mitigate
this challenge, various methods have been proposed to enhance the FL training
process. This paper endeavors to tackle the issue of data heterogeneity from
another perspective -- by improving FL algorithms prior to the actual training
stage. Specifically, we introduce the Client2Vec mechanism, which generates a
unique client index for each client before the commencement of FL training.
Subsequently, we leverage the generated client index to enhance the subsequent
FL training process. To demonstrate the effectiveness of the proposed
Client2Vec method, we conduct three case studies that assess the impact of the
client index on the FL training process. These case studies encompass enhanced
client sampling, model aggregation, and local training. Extensive experiments
conducted on diverse datasets and model architectures show the efficacy of
Client2Vec across all three case studies. Our code is avaliable at
\url{https://github.com/LINs-lab/client2vec}.
