  Entropy production plays a fundamental role in nonequilibrium thermodynamics
to quantify the irreversibility of open systems. Its positivity can be ensured
for a wide class of setups, but the entropy production rate can become negative
sometimes. This is often taken as an indicator of non-Markovian dynamics. We
make this link precise by showing under which conditions a negative entropy
production rate implies non-Markovian dynamics and when it does not. This is
established within a unified language for two setups: (i) the dynamics
resulting from a coarse-grained description of a system in contact with a
single heat bath described by a Markovian master equation and (ii) the
classical Hamiltonian dynamics of a driven system, which is coupled arbitrary
strongly to a single heat bath. The quantum version of the latter result is
shown not to hold despite the fact that the integrated thermodynamic
description is formally equivalent to the classical case. The instantaneous
steady state of a non-Markovian dynamics plays an important element in our
study. Our key contribution is to provide a consistent theoretical framework to
study the finite-time thermodynamics of a large class of dynamics with a
precise link to its non-Markovianity.
