  We have used an {\it ab initio} Gamow shell model to study the isospin
symmetry breaking in the $A=16$ mirror nuclei of $^{16}$F, $^{16}$N, $^{16}$Ne
and $^{16}$C. Starting from a chiral interaction with two-nucleon force (2NF)
at N$^3$LO and three-nucleon force (3NF) at N$^2$LO, a complex-momentum ${\it
psd}$-shell Hamiltonian was constructed by employing the many-body perturbation
theory in the Gamow Hartree-Fock basis which includes bound, resonant and
continuum states self-consistently. Such an elaborated {\it ab initio} Gamow
shell model with both continuum coupling and 3NF included can properly treat
the many-body correlations of weakly bound and unbound nuclei. The mirror
partners of $^{16}$F and $^{16}$N exhibit different level orders in their
excitation spectra, which can be well explained by the inclusion of 3NF in the
calculation. The isospin asymmetry between the mirror partners $^{16}$Ne and
$^{16}$C was studied in detail by insight into their configuration structures.
The interplay between 3NF and the continuum coupling is discussed in the weakly
bound and unbound nuclear states.
