  We study the symmetrized noncommutative arithmetic geometric mean inequality
introduced(AGM) by Recht and R\'{e} $$
  \|\frac{(n-d)!}{n!}\sum\limits_{{ j_1,...,j_d \mbox{ different}}
}A_{j_{1}}^*A_{j_{2}}^*...A_{j_{d}}^*A_{j_{d}}...A_{j_{2}}A_{j_{1}} \| \leq
C(d,n) \|\frac{1}{n} \sum_{j=1}^n A_j^*A_j\|^d .$$
  Complementing the results from Recht and R\'{e}, we find upper bounds for
C(d,n) under additional assumptions. Moreover, using free probability, we show
that $C(d, n) > 1$, thereby disproving the most optimistic conjecture from
Recht and R\'{e}.We also prove a deviation result for the symmetrized-AGM
inequality which shows that the symmetric inequality almost holds for many
classes of random matrices. Finally we apply our results to the incremental
gradient method(IGM).
