  We revisit the idea of generating the Higgs mu parameter through a
spontaneously broken Peccei-Quinn (PQ) symmetry in gauge-mediated supersymmetry
breaking scenario. For the messenger scale of gauge mediation higher than the
PQ scale, the setup naturally generates mu ~ m_soft and the Higgs soft
parameter B \leq O(m_soft) with the CP phase of B aligned to the phase of
gaugino masses, while giving the PQ scale v_PQ ~ \surd (m_soft {\Lambda}),
where m_soft denote the gauge-mediated gaugino or sfermion masses and {\Lambda}
is the cutoff scale which can be identified as the Planck scale or the GUT
scale. The PQ sector of the model results in distinctive cosmology including a
late thermal inflation. We discuss the issue of dark matter and baryogenesis in
the resulting thermal inflation scenario, and find that a right amount of
gravitino dark matter can be produced together with a successful Affleck-Dine
leptogenesis, when the gravitino mass m_{3/2} = O(100) keV.
