  We study the problem of finding orthogonal low-rank approximations of
symmetric tensors. In the case of matrices, the approximation is a truncated
singular value decomposition which is then symmetric. Moreover, for rank-one
approximations of tensors of any dimension, a classical result proven by Banach
in 1938 shows that the optimal approximation can always be chosen to be
symmetric. In contrast to these results, this article shows that the
corresponding statement is no longer true for orthogonal approximations of
higher rank. Specifically, for any of the four common notions of tensor
orthogonality used in the literature, we show that optimal orthogonal
approximations of rank greater than one cannot always be chosen to be
symmetric.
