  We propose a physical mechanism for inverse magnetic catalysis, the
suppression of the chiral condensate by an external magnetic field in QCD
around the critical temperature. We show that this effect, seen in lattice
simulations, is a result of how the sea quarks react to the magnetic field. We
find that the suppression of the condensate happens because the quark
determinant can suppress low quark modes by ordering the Polyakov loop. This
mechanism is particularly efficient around $T_c$ where the Polyakov loop
effective potential is flat and the determinant can have a significant ordering
effect. Our picture suggests that for the description of QCD in large magnetic
fields it is crucial to properly capture the interaction between the Polyakov
loop and the sea quarks, both in low-energy effective models and on the
lattice.
