  Time in-homogeneous cyclic Markov chain Monte Carlo (MCMC) samplers,
including deterministic scan Gibbs samplers and Metropolis within Gibbs
samplers, are extensively used for sampling from multi-dimensional
distributions. We establish a multivariate strong invariance principle (SIP)
for Markov chains associated with these samplers. The rate of this SIP
essentially aligns with the tightest rate available for time homogeneous Markov
chains. The SIP implies the strong law of large numbers (SLLN) and the central
limit theorem (CLT), and plays an essential role in uncertainty assessments.
Using the SIP, we give conditions under which the multivariate batch means
estimator for estimating the covariance matrix in the multivariate CLT is
strongly consistent. Additionally, we provide conditions for a multivariate
fixed volume sequential termination rule, which is associated with the concept
of effective sample size (ESS), to be asymptotically valid. Our uncertainty
assessment tools are demonstrated through various numerical experiments.
