  We investigate the phase diagram and thermodynamics of $SU(N)$ pure
Yang-Mills theory on a manifold $\mathbb{T}^2\times \mathbb{R}^2$ with an
effective model that includes two Polyakov loops along two compactified
directions. We find that a rich phase structure can appear owing to the
spontaneous breaking of two center symmetries for $N=2$ and $3$. Thermodynamic
quantities are obtained in the model and compared with recent lattice results.
It is shown that two Polyakov loops play significant roles in thermodynamics on
$\mathbb{T}^2\times \mathbb{R}^2$.
