  The crystal structure of the PbMg$_{1/3}$Ta$_{2/3}$O$_3$ (PMT) relaxor
ferroelectric was studied under hydrostatic pressure up to $\sim 7$ GPa by
means of powder neutron diffraction. We find a drastic pressure-induced
decrease of the lead displacement from the inversion centre which correlates
with an increase by $\sim$ 50 % of the anisotropy of the oxygen temperature
factor. The vibrations of the Mg/Ta are, in contrast, rather pressure
insensitive. We attribute these changes being responsible for the previously
reported pressure-induced suppression of the anomalous dielectric permittivity
and diffuse scattering in relaxor ferroelectrics.
