  We prove \emph{optimal} improvements of the Hardy inequality on the
hyperbolic space. Here, optimal means that the resulting operator is critical
in the sense of [J.Funct.Anal. 266 (2014), pp. 4422-89], namely the associated
inequality cannot be further improved. Such inequalities arise from more
general, \emph{optimal} ones valid for the operator $ P_{\lambda}:=
-\Delta_{\mathbb{H}^N} - \lambda$ where $0 \leq \lambda \leq
\lambda_{1}(\mathbb{H}^N)$ and $\lambda_{1}(\mathbb{H}^N)$ is the bottom of the
$L^2$ spectrum of $-\Delta_{\mathbb{H}^N} $, a problem that had been studied in
[J.Funct.Anal. 272 (2017), pp. 1661-1703 ] only for the operator
$P_{\lambda_{1}(\mathbb{H}^N)}$. A different, critical and new inequality on
$\mathbb{H}^N$, locally of Hardy type, is also shown. Such results have in fact
greater generality since there are shown on general Cartan-Hadamard manifolds
under curvature assumptions, possibly depending on the point.
Existence/nonexistence of extremals for the related Hardy-Poincar\'e
inequalities are also proved using concentration-compactness technique and a
Liouville comparison theorem. As applications of our inequalities we obtain an
improved Rellich inequality and we derive a quantitative version of
Heisenberg-Pauli-Weyl uncertainty principle for the operator $P_\lambda.$
