  We perform a combined experimental and theoretical study of a magnetic-field
($B$) induced evolution of magnetic and ferroelectric properties in an
antiferromagnetic material Pb(TiO)Cu$_4$(PO$_4$)$_4$, whose structure is
characterized by a staggered array of Cu$_4$O$_{12}$ magnetic units with convex
geometry known as square cupola. Our experiments show a $B$-induced phase
transition from a previously reported low-$B$ linear magnetoelectric phase to a
new high-$B$ magnetoelectric phase, which accompanies a 90$^\circ$ flop of
electric polarization and gigantic magnetodielectric effect. Moreover, we
observe a $B$-induced sign reversal of ferroelectric polarization in the
high-$B$ phase. Our model and first-principles calculations reveal that the
observed complex magnetoelectric behavior is well explained in terms of a
$B$-dependent electric polarization generated in each Cu$_4$O$_{12}$ unit by
the so-called exchange striction mechanism. The present study demonstrates that
the materials design based on the magnetic structural unit with convex geometry
deserves to be explored for developing strong magnetoelectric couplings.
