  Production MPI codes need checkpoint-restart (CPR) support. Clearly,
checkpoint-restart libraries must be fault tolerant lest they open up a window
of vulnerability for failures with byzantine outcomes. But, certain popular
libraries that leverage MPI are evidently not fault tolerant. Nowadays, fault
detection with automatic recovery without batch requeueing is a strong
requirement for production environments. Thus, allowing deadlock and setting
long timeouts are suboptimal for fault detection even when paired with
conservative recovery from the penultimate checkpoint.
  When MPI is used as a communication mechanism within a CPR library, such
libraries must offer fault-tolerant extensions with minimal detection,
isolation, mitigation, and potential recovery semantics to aid the CPR's
library fail-backward. Communication between MPI and the checkpoint library
regarding system health may be valuable. For fault-tolerant MPI programs (e.g.,
using APIs like FA-MPI, Stages/Reinit, or ULFM), the checkpoint library must
cooperate with the extended model or else invalidate fault-tolerant operation.
