  Unitary equivariance is a natural symmetry that occurs in many contexts in
physics and mathematics. Optimization problems with such symmetry can often be
formulated as semidefinite programs for a $d^{p+q}$-dimensional matrix variable
that commutes with $U^{\otimes p} \otimes \bar{U}^{\otimes q}$, for all $U \in
\mathrm{U}(d)$. Solving such problems naively can be prohibitively expensive
even if $p+q$ is small but the local dimension $d$ is large. We show that,
under additional symmetry assumptions, this problem reduces to a linear program
that can be solved in time that does not scale in $d$, and we provide a general
framework to execute this reduction under different types of symmetries. The
key ingredient of our method is a compact parametrization of the solution space
by linear combinations of walled Brauer algebra diagrams. This parametrization
requires the idempotents of a Gelfand-Tsetlin basis, which we obtain by
adapting a general method arXiv:1606.08900 inspired by the Okounkov-Vershik
approach. To illustrate potential applications, we use several examples from
quantum information: deciding the principal eigenvalue of a quantum state,
quantum majority vote, asymmetric cloning and transformation of a black-box
unitary. We also outline a possible route for extending our method to general
unitary-equivariant semidefinite programs.
