  We investigate the influence of random variations of the Galactic
gravitational field on the apparent celestial positions of extragalactic
sources. The basic statistical characteristics of a stochastic process
(first-order moments, an autocorrelation function and a power spectral density)
are used to describe a light ray deflection in a gravitational field of
randomly moving point masses as a function of the source coordinates. We map a
2D distribution of the standard deviation of the angular shifts in positions of
distant sources (including reference sources of the International Celestial
Reference Frame) with respect to their true positions. For different Galactic
matter distributions the standard deviation of the offset angle can reach
several tens of $\mu as$ (microarcsecond) toward the Galactic center,
decreasing down to 4-6 $\mu as$ at high galactic latitudes. The conditional
standard deviation (`jitter') of 2.5 $\mu as$ is reached within 10 years at
high galactic latitudes and within a few months toward the inner part of the
Galaxy. The photometric microlensing events are not expected to be disturbed by
astrometric random variations anywhere except the inner part of the Galaxy as
the Einstein--Chvolson times are typically much shorter than the jittering
timescale. While a jitter of a single reference source can be up to dozens of
$\mu as$ over some reasonable observational time, using a sample of reference
sources would reduce the error in relative astrometry. The obtained results can
be used for estimating the physical upper limits on the time-dependent accuracy
of astrometric measurements.
