  We identify the dimension of the centralizer of the symmetric group
$\mathfrak{S}_d$ in the partition algebra $\mathcal{A}_d(\delta)$ and in the
Brauer algebra $\mathcal{B}_d(\delta)$ with the number of multidigraphs with
$d$ arrows and the number of disjoint union of directed cycles with $d$ arrows,
respectively. Using Schur-Weyl duality as a fundamental theory, we conclude
that each centralizer is related with the $G$-invariant space
$P^d(M_n(\mathbf{k}))^G$ of degree $d$ homogeneous polynomials on $n \times n$
matrices, where $G$ is the orthogonal group and the group of permutation
matrices, respectively. Our approach gives a uniform way to show that the
dimensions of $P^d(M_n(\mathbf{k}))^G$ are stable for sufficiently large $n$.
