  We examine rectangular W-algebras with $so(M)$ or $sp(2M)$ symmetry, which
can be realized as the asymptotic symmetry of higher spin gravities with
restricted matrix extensions. We compute the central charges of the algebras
and the levels of $so(M)$ or $sp(2M)$ affine subalgebras by applying the
Hamiltonian reductions of $so$ or $sp$ type Lie algebras. For simple cases with
generators of spin up to two, we obtain their operator product expansions by
requiring the associativity. We further claim that the W-algebras can be
realized as the symmetry algebras of dual coset CFTs and provide several strong
supports. The analysis can be regarded as a check of extended higher spin
holographies including full quantum corrections. We also extend the analysis by
introducing $\mathcal{N}=1$ supersymmetry.
