  Gaia used a large sample of photometrically selected active galactic nuclei
(AGNs) and quasars to remove the residual spin of its global proper motion
system in order to achieve a maximally inertial reference frame. A small
fraction of these reference objects have statistically significant astrometric
proper motions in Gaia EDR3. We compile a source sample of $105,593$
high-fidelity AGNs with accurate spectroscopically determined redshifts above
0.5 from the SDSS and normalized proper motions below 4. The rate of genuinely
perturbed proper motions is at least 0.17\%. A smaller high completeness sample
of 152 quasars with excess proper motions at a confidence level of 0.9995 is
examined in detail. Pan-STARRS images and Gaia-resolved pairs reveal that 29\%
of the sample are either double sources or gravitationally lensed quasars. An
Anderson--Darling test on parameters of a smaller high-reliability sample and
their statistical controls reveals 17 significant factors that favor
multiplicity and multi-source structure as the main cause of perturbed
astrometry. Using a nearest neighbor distance statistical analysis and counts
of close companions in Gaia on a much larger initial sample of AGNs, an excess
of closely separated sources in Gaia is detected. At least 0.33\% of all
optical quasars are genuinely double or multiply imaged. We provide a list of
44 candidate double or multiple AGNs and four previously known gravitational
lenses. Many proper motion quasars may be more closely separated, unresolved
doubles exhibiting the variability imposed motion (VIM) effect, and a smaller
fraction may be chance alignments with foreground stars causing weak
gravitational lensing.
