  Kitaev's quantum double model is a lattice gauge theoretic realization of
Dijkgraaf-Witten topological quantum field theory (TQFT), its topologically
protected ground state space has broad applications for topological quantum
computation and topological quantum memory. We investigate the $\mathbb{Z}_2$
symmetry enriched generalization of the model for the cyclic Abelian group in a
categorical framework and present an explicit Hamiltonian construction. This
model provides a lattice realization of the $\mathbb{Z}_2$ symmetry enriched
topological (SET) phase. We discuss in detail the categorical symmetry of the
phase, for which the electric-magnetic (EM) duality symmetry is a special case.
The aspects of symmetry defects are investigated using the $G$-crossed unitary
braided fusion category (UBFC). By determining the corresponding anyon
condensation, the gapped boundaries and boundary-bulk duality are also
investigated. Then we carefully construct the explicit lattice realization of
EM duality for these SET phases.
