  Moir\'e heterobilayer transition metal dichalcogenides (TMDs) emerge as an
ideal system for simulating the single-band Hubbard model and interesting
correlated phases have been observed in these systems. Nevertheless, the
moir\'e bands in heterobilayer TMDs were believed to be topologically trivial.
Recently, it was reported that both a quantum valley Hall insulating state at
filling $\nu=2$ (two holes per moir\'e unit cell) and a valley-polarized
quantum anomalous Hall state at filling $\nu=1$ were observed in AB stacked
moir\'e MoTe$_2$/WSe$_2$ heterobilayers. However, how the topologically
nontrivial states emerge is not known. In this work, we propose that the
pseudo-magnetic fields induced by lattice relaxation in moir\'e
MoTe$_2$/WSe$_2$ heterobilayers could naturally give rise to moir\'e bands with
finite Chern numbers. We show that a time-reversal invariant quantum valley
Hall insulator is formed at full-filing $\nu=2$, when two moir\'e bands with
opposite Chern numbers are filled. At half-filling $\nu=1$, Coulomb interaction
lifts the valley degeneracy and results in a valley-polarized quantum anomalous
Hall state, as observed in the experiment. Our theory identifies a new way to
achieve topologically non-trivial states in heterobilayer TMD materials.
