  We develop a generic $\mathbf{k}\cdot \mathbf{p}$ open momentum space method
for calculating the Hofstadter butterfly of both continuum (Moir\'e) models and
tight-binding models, where the quasimomentum is directly substituted by the
Landau level (LL) operators. By taking a LL cutoff (and a reciprocal lattice
cutoff for continuum models), one obtains the Hofstadter butterfly with in-gap
spectral flows. For continuum models such as the Moir\'e model for twisted
bilayer graphene, our method gives a sparse Hamiltonian, making it much more
efficient than existing methods. The spectral flows in the Hofstadter gaps can
be understood as edge states on a momentum space boundary, from which one can
determine the two integers ($t_\nu,s_\nu$) of a gap $\nu$ satisfying the
Diophantine equation. The spectral flows can also be removed to obtain a clear
Hofstadter butterfly. While $t_\nu$ is known as the Chern number, our theory
identifies $s_\nu$ as a dual Chern number for the momentum space, which
corresponds to a quantized Lorentz susceptibility $\gamma_{xy}=eBs_\nu$.
