  Topological magnon insulators in insulating Kagom\'e ferromagnets have been
extensively studied in a series of papers. It has been established that
Dzyaloshinskii-Moriya interaction (DMI) is the key ingredient to observe a
nontrivial topological magnon with edge modes. However, insulating
antiferromagnets on the Kagom\'e lattice are frustrated systems considered as a
playground for studying quantum spin liquid physics. In these systems the DMI
can induce a coplanar but noncollinear magnetic orders with a $\mathbf{q}=0$
propagating wavevector. We show that topological magnon bands are absent in
this coplanar spin texture in sharp contrast to collinear ferromagnets with
DMI. Hence, geometrically frustrated Kagom\'e antiferromagnets can be deemed
topologically trivial. The presence of an out-of-plane magnetic field in these
frustrated magnets induces noncoplanar spin textures exhibiting a nonzero spin
scalar chirality. We show that the field-induced spin chirality provides
topological magnon bands in Kagom\'e antiferromagnets without the need of DMI
and survives in the chiral spin liquid phase of frustrated magnets. Possible
experimentally material includes iron jarosite KFe$_3$(OH)$_{6}$(SO$_{4}$)$_2$.
