  We report the observation of two-fold rotation symmetry in charge density
wave (CDW) state in the newly discovered Kagome superconductor CsV$_3$Sb$_5$.
Below its CDW transition temperature ($T_{CDW}$), the polarization rotation of
the reflected laser beam promptly emerges and increases close to about 1 mrad,
and the rotation angle shows two-fold rotation symmetry. With femtosecond laser
pulse pumping, the rotation angle can be easily suppressed and then recovers in
several picoseconds accompanied with coherent oscillations. Significantly, the
oscillations in the signal also experience a 180 degree periodic change. Our
investigation provides clear optical evidence for the formation of nematic
order with two-fold rotation symmetry just below $T_{CDW}$. The results imply a
immediate development of nematicity and possible time-reversal symmetry
breaking in CDW state of CsV$_3$Sb$_5$.
