  The effect of quark anomalous magnetic moment (AMM) to chiral restoration and
deconfinement phase transitions in baryon chemical potential-temperature
$(\mu_B-T)$ plane under magnetic fields is investigated in frame of a
Pauli-Villars regularized PNJL model. It's found that the quark AMM plays the
role of inverse catalysis to the phase transitions, and large quark AMM will
change the magnetic catalysis phenomena of phase transitions to inverse
magnetic catalysis in the whole $\mu_B-T$ plane. For a fixed magnetic field,
the critical temperature $T_c$ and critical baryon chemical potential $\mu_B^c$
decreases with quark AMM. The stronger the magnetic field is, the inverse
catalysis effect of AMM becomes more important. For a small AMM
$\kappa=\kappa_1$, it shows the magnetic catalysis effect for critical
temperature $T_c$ at vanishing $\mu_B$ with increasing magnetic field, and
(inverse) magnetic catalysis effect for critical baryon chemical potential
$\mu_B^c$ at vanishing $T$ under (weak) strong magnetic field. At finite $T$
and $\mu_B$, there exist some crossings of the phase transition lines with
different magnetic field. For a large AMM $\kappa=\kappa_2$, we obtain the
inverse magnetic catalysis effect in the whole $\mu_B-T$ plane, and no
crossings of phase transition lines happen.
