  Regardless of studies and debates over a century, the statistical origin of
the second law of thermodynamics still remains illusive. One essential obstacle
is the lack of a proper theoretical formalism for non-equilibrium entropy. Here
I revisit the seminal ideas about non-equilibrium statistical entropy due to
Boltzmann and due to Gibbs, and synthesize them into a coherent and precise
framework. Using this framework, I clarify the anthropomorphic principle of
entropy, and analyze the evolution of entropy for classical Hamiltonian systems
under different experimental setups. I find that evolution of Boltzmann entropy
obeys a Stochastic H-Theorem, which relates probability of Boltzmann entropy
increasing to that of decreasing. By contrast, the coarse-grained Gibbs entropy
is monotonically increasing, if the microscopic dynamics is locally mixing, and
the initial state is a Boltzmann state. These results clarify the precise
meaning of the second law of thermodynamics for classical systems, and
demonstrate that it is the initial condition as a Boltzmann state that is
ultimately responsible for the arrow of time.
