  We develop rigorous estimates and provably convergent approximations for the
memory integral in the Mori-Zwanzig (MZ) formulation. The new theory is built
upon rigorous mathematical foundations and is presented for both state-space
and probability density function space formulations of the MZ equation. In
particular, we derive errors bounds and sufficient convergence conditions for
short-memory approximations, the $t$-model, and hierarchical (finite-memory)
approximations. In addition, we derive computable upper bounds for the MZ
memory integral, which allow us to estimate (a priori) the contribution of the
MZ memory to the dynamics. Numerical examples demonstrating convergence of the
proposed algorithms are presented for linear and nonlinear dynamical systems
evolving from random initial states.
