  Plasmons are usually described in terms of macroscopic quantities such as
electric fields and currents. However as fundamental excitations of metals they
are also quantum objects with internal structure. We demonstrate that this can
induce an intrinsic dipole moment which is tied to the quantum geometry of the
Hilbert space of plasmon states. This {\it quantum geometric dipole} offers a
unique handle for manipulation of plasmon dynamics, via density modulations and
electric fields. As a concrete example we demonstrate that scattering of
plasmons with non-vanishing quantum geometric dipole from impurities is
non-reciprocal, skewing in different directions in a valley-dependent fashion.
This internal structure can be used to control plasmon trajectories in two
dimensional materials.
