  The $N$th power of a polynomial matrix of fixed size and degree can be
computed by binary powering as fast as multiplying two polynomials of linear
degree in~$N$. When Fast Fourier Transform (FFT) is available, the resulting
complexity is \emph{softly linear} in~$N$, i.e.~linear in~$N$ with extra
logarithmic factors. We show that it is possible to beat binary powering, by an
algorithm whose complexity is \emph{purely linear} in~$N$, even in absence of
FFT. The key result making this improvement possible is that the entries of the
$N$th power of a polynomial matrix satisfy linear differential equations with
polynomial coefficients whose orders and degrees are independent of~$N$.
Similar algorithms are proposed for two related problems: computing the $N$th
term of a C-finite sequence of polynomials, and modular exponentiation to the
power $N$ for bivariate polynomials.
