  The thermodynamic formalism allows one to access the chaotic properties of
equilibrium and out-of-equilibrium systems, by deriving those from a dynamical
partition function. The definition that has been given for this partition
function within the framework of discrete time Markov chains was not suitable
for continuous time Markov dynamics. Here we propose another interpretation of
the definition that allows us to apply the thermodynamic formalism to
continuous time.
  We also generalize the formalism --a dynamical Gibbs ensemble construction--
to a whole family of observables and their associated large deviation
functions. This allows us to make the connection between the thermodynamic
formalism and the observable involved in the much-studied fluctuation theorem.
  We illustrate our approach on various physical systems: random walks,
exclusion processes, an Ising model and the contact process. In the latter
cases, we identify a signature of the occurrence of dynamical phase
transitions. We show that this signature can already be unravelled using the
simplest dynamical ensemble one could define, based on the number of
configuration changes a system has undergone over an asymptotically large time
window.
