  We explore two differentiable deep declarative layers, namely least squares
on sphere (LESS) and implicit eigen decomposition (IED), for learning the
principal matrix features (PMaF). It can be used to represent data features
with a low-dimensional vector containing dominant information from a
high-dimensional matrix. We first solve the problems with iterative
optimization in the forward pass and then backpropagate the solution for
implicit gradients under a bi-level optimization framework. Particularly,
adaptive descent steps with the backtracking line search method and descent
decay in the tangent space are studied to improve the forward pass efficiency
of LESS. Meanwhile, exploited data structures are used to greatly reduce the
computational complexity in the backward pass of LESS and IED. Empirically, we
demonstrate the superiority of our layers over the off-the-shelf baselines by
comparing the solution optimality and computational requirements.
