  Orthogonal matrices play an important role in probability and statistics,
particularly in high-dimensional statistical models. Parameterizing these
models using orthogonal matrices facilitates dimension reduction and parameter
identification. However, establishing the theoretical validity of statistical
inference in these models from a frequentist perspective is challenging,
leading to a preference for Bayesian approaches because of their ability to
offer consistent uncertainty quantification. Markov chain Monte Carlo methods
are commonly used for numerical approximation of posterior distributions, and
sampling on the Stiefel manifold, which comprises orthogonal matrices, poses
significant difficulties. While various strategies have been proposed for this
purpose, gradient-based Markov chain Monte Carlo with parameterizations is the
most efficient. However, a comprehensive comparison of these parameterizations
is lacking in the existing literature. This study aims to address this gap by
evaluating numerical efficiency of the four alternative parameterizations of
orthogonal matrices under equivalent conditions. The evaluation was conducted
for four problems. The results suggest that polar expansion parameterization is
the most efficient, particularly for the high-dimensional and complex problems.
However, all parameterizations exhibit limitations in significantly
high-dimensional or difficult tasks, emphasizing the need for further
advancements in sampling methods for orthogonal matrices.
