  Frequency is one of the major factors for training quality word embeddings.
Several work has recently discussed the stability of word embeddings in general
domain and suggested factors influencing the stability. In this work, we
conduct a detailed analysis on the stability of concept embeddings in medical
domain, particularly the relation with concept frequency. The analysis reveals
the surprising high stability of low-frequency concepts: low-frequency (<100)
concepts have the same high stability as high-frequency (>1000) concepts. To
develop a deeper understanding of this finding, we propose a new factor, the
noisiness of context words, which influences the stability of medical concept
embeddings, regardless of frequency. We evaluate the proposed factor by showing
the linear correlation with the stability of medical concept embeddings. The
correlations are clear and consistent with various groups of medical concepts.
Based on the linear relations, we make suggestions on ways to adjust the
noisiness of context words for the improvement of stability. Finally, we
demonstrate that the proposed factor extends to the word embedding stability in
general domain.
