  We have studied the evolution with temperature of the local as well as the
average crystal structure of MgB$_2$ using the real-space atomic pair
distribution function (PDF) measured by high resolution neutron powder
diffraction. We have investigated the correlations of the B-B and B-Mg nearest
neighbor pair motion by comparing, in the wide temperature range from T=10 K up
to T=600 K, the mean-square displacements (MSD) of single atoms with the
mean-square relative displacements (MSRD) obtained from the PDF peak
linewidths. The results show that the single atom B and Mg vibrations are
mostly decoupled from each other, with a small predominance of positive (in
phase) correlation factor for both the B-B and B-Mg pairs. The small positive
correlation is almost temperature independent, in contrast with our theoretical
calculations; this can be a direct consequence of the strong decay processes of
the $E_{2g}$ anharmonic phonons.
