  Quantum anomalous Hall (QAH) effect in the films with nontrivial band
structure accompanies the ferromagnetic transition in the system of magnetic
dopants. Experimentally, the QAH transition manifests itself as a jump in the
dependence of longitudinal resistivity on a weak external magnetic field.
Microscopically, this jump originates from the emergence of a chiral edge mode
on one side of the ferromagnetic transition. We study analytically the effect
of an extended confinement on the structure of the edge modes. We employ the
simplest model of the extended confinement in the form of potential step next
to the hard wall. It is shown that, unlike the conventional quantum Hall
effect, where all edge channels are chiral, in QAH effect, a complex structure
of the boundary leads to nonchiral edge modes which are present on both sides
of the ferromagnetic transition. Wave functions of nonchiral modes are
different above and below the transition: on the "topological" side, where the
chiral edge mode is supported, nonchiral modes are "repelled" from the
boundary, i.e. they are much less localized than on the "trivial" side. Thus,
the disorder-induced scattering into these modes will boost the extension of
the chiral edge mode. The prime experimental manifestation of nonchiral modes
is that, by contributing to longitudinal resistance, they smear the QAH
transition.
