  Time-reversal symmetry of microscopic laws dictates that the equilibrium
distribution of a stochastic process must obey the detailed balance. On the
other hand, cyclic Markov processes that do not admit equilibrium distributions
with detailed balance, are often used to model open systems driven out of
equilibrium by external agents. I show that for a Markov model without detailed
balance, an extended Markov model that explicitly includes the degrees of
freedom for the driving agent can be constructed, such that the original cyclic
Markov model for the driven system can be recovered as an approximation at
early times, by summing over the degrees of freedom for the driving agent. In
the process, the widely accepted formula for the entropy production in a cyclic
Markov model is explicitly expressed as a time derivative of an entropy
component in the extended model. I also find an analytic formula for the
entropy component that is hidden in the cyclic Markov model.
