  Berry phase physics is closely related to a number of topological states of
matter. Recently discovered topological semimetals are believed to host a
nontrivial $\pi$ Berry phase to induce a phase shift of $\pm 1/8$ in the
quantum oscillation ($+$ for hole and $-$ for electron carriers). We
theoretically study the Shubnikov-de Haas oscillation of Weyl and Dirac
semimetals, taking into account their topological nature and inter-Landau band
scattering. For a Weyl semimetal with broken time-reversal symmetry, the phase
shift is found to change nonmonotonically and go beyond known values of $\pm
1/8$ and $\pm 5/8$. For a Dirac semimetal or paramagnetic Weyl semimetal,
time-reversal symmetry leads to a discrete phase shift of $\pm 1/8$ or $\pm
5/8$, as a function of the Fermi energy. Different from the previous works, we
find that the topological band inversion can lead to beating patterns in the
absence of Zeeman splitting. We also find the resistivity peaks should be
assigned integers in the Landau index plot. Our findings may account for recent
experiments in Cd$_2$As$_3$ and should be helpful for exploring the Berry phase
in various 3D systems.
