  Crystalline materials are promising candidates as substrates or
high-reflective coatings of mirrors to reduce thermal noises in future laser
interferometric gravitational wave detectors. However, birefringence of such
materials could degrade the sensitivity of gravitational wave detectors, not
only because it can introduce optical losses, but also because its fluctuations
create extra phase noise in the arm cavity reflected beam. In this paper, we
analytically estimate the effects of birefringence and its fluctuations in the
mirror substrate and coating for gravitational wave detectors. Our calculations
show that the requirements for the birefringence fluctuations in silicon
substrate and AlGaAs coating will be on the order of $10^{-8}$ and $10^{-10}$
rad/$\sqrt{\rm Hz}$ at 100~Hz, respectively, for future gravitational wave
detectors. We also point out that optical cavity response needs to be carefully
taken into account to estimate optical losses from depolarization.
