  The Dirac electrons occupying the surface states (SSs) of topological
insulators (TIs) have been predicted to exhibit many exciting magneto-transport
phenomena. Here we report on the first experimental observation of an
unconventional planar Hall effect (PHE) and an electrically gate-tunable
hysteretic planar magnetoresistance (PMR) in EuS/TI heterostructures, in which
EuS is a ferromagnetic insulator (FMI) with an in-plane magnetization. In such
exchange-coupled FMI/TI heterostructures, we find a significant (suppressed)
PHE when the in-plane magnetic field is parallel (perpendicular) to the
electric current. This behavior differs from previous observations of the PHE
in ferromagnets and semiconductors. Furthermore, as the thickness of the 3D TI
films is reduced into the 2D limit, in which the Dirac SSs develop a
hybridization gap, we find a suppression of the PHE around the charge neutral
point indicating the vital role of Dirac SSs in this phenomenon. To explain our
findings, we outline a symmetry argument that excludes linear-Hall mechanisms
and suggest two possible non-linear Hall mechanisms that can account for all
the essential qualitative features in our observations.
