  A novel algorithm for designing optimized orthonormal transform-matrix
codebooks for adaptive transform coding of a non-stationary vector process is
proposed. This algorithm relies on a block-wise stationary model of a
non-stationary process and finds a codebook of transform-matrices by minimizing
the end-to-end mean square error of transform coding averaged over the
distribution of stationary blocks of vectors. The algorithm, which belongs to
the class of block-coordinate descent algorithms, solves an intermediate
minimization problem involving matrix-orthonormality constraints in a
computationally efficient manner by mapping the problem from the Euclidean
space to the Stiefel manifold. As such, the algorithm can be broadly applied to
any adaptive transform coding problem. Preliminary results obtained with
inter-prediction residuals in an H265 video codec are presented to demonstrate
the advantage of optimized adaptive transform codes over non-adaptive codes
based on the standard DCT.
