  Calcium fluoride (CaF$_2$) crystalline whispering gallery mode resonators
(WGMRs) exhibit ultrahigh intrinsic quality factors and a low power anomalous
dispersion in the communication and mid-infrared bands, making them attractive
platforms for microresonator-based comb generation. However, their unique
negative thermo-optic effects pose challenges when achieving thermal
equilibrium. To our knowledge, our experiments serve as the first demonstration
of soliton microcombs in Q > 109 CaF$_2$ WGMRs. We observed soliton
mode-locking and bidirectional switching of soliton numbers caused by the
negative thermo-optic effects. Additionally, various soliton formation dynamics
are shown, including breathing and vibrational solitons, which can be
attributed to thermo-photomechanical oscillations. Thus, our results enrich the
soliton generation platform and provide a reference for generating solitons
from WGMRs that comprise other materials with negative thermo-optic effects. In
the future, the ultrahigh quality factor of CaF$_2$ crystal cavities may enable
the generation of sub-milliwatt-level broad-spectrum soliton combs.
