  There has been intense recent interest in the two-dimensional electron gases
(2DEGs) that form at the surfaces and interfaces of KTaO$_3$ (KTO), with the
discovery of superconductivity at temperatures significantly higher than those
of similar 2DEGs based on SrTiO$_3$ (STO). Here we demonstrate that KTO 2DEGs
fabricated under conditions that suppress the superconductivity show a large
spontaneous Hall effect at low temperatures. The transverse response is
asymmetric in an applied perpendicular magnetic field and becomes hysteretic at
millikelvin temperatures. The hysteresis is due to long range magnetic order
arising from local Ta$^{4+}$ moments. However, the most striking features of
the data are the asymmetry of the transverse response and the large spontaneous
transverse resistance at zero field, which can be a significant fraction of the
longitudinal resistance and depends on crystal orientation. Both effects are
due to the presence of a dominant contribution to the transverse response that
is symmetric in perpendicular field, suggesting that its origin is topological
in nature. We argue that this contribution arises from Berry curvature dipoles
coupled with nonequilibrium conditions induced by the measuring current.
