  We generalize the convex duality symmetry in Gibbs' statistical ensemble
formulation, between Massieu's free entropy $\Phi_{V,N} (\beta)$ and the Gibbs
entropy $\varphi_{V,N}(u)$ as a function of mean internal energy $u$. The
duality tells us that Gibbs thermodynamic entropy is to the law of large
numbers (LLN) for arithmetic sample means what Shannon's information entropy is
to the LLN for empirical counting frequencies. Following the same logic, we
identify $u$ as the conjugate variable to counting frequency, a Hamilton-Jacobi
equation for Shannon entropy as an equation of state, and suggest an eigenvalue
problem for modeling statistical frequencies of correlated data.
