  By computing the operator product expansions between the first two ${\cal
N}=4$ higher spin multiplets in the unitary coset model, the (anti)commutators
of higher spin currents are obtained under the large $(N,k)$ 't Hooft-like
limit. The free field realization with complex bosons and fermions is
presented. The (anti)commutators for generic spins $s_1$ and $s_2$ with
manifest $SO(4)$ symmetry at vanishing 't Hooft-like coupling constant are
completely determined. The structure constants can be written in terms of the
ones in the ${\cal N}=2$ ${\cal W}_{\infty}$ algebra found by Bergshoeff, Pope,
Romans, Sezgin and Shen previously, in addition to the spin-dependent
fractional coefficients and two $SO(4)$ invariant tensors. We also describe the
${\cal N}=4$ higher spin generators, by using the above coset construction
results, for general super spin $s$ in terms of oscillators in the matrix
generalization of $AdS_3$ Vasiliev higher spin theory at nonzero 't Hooft-like
coupling constant. We obtain the ${\cal N}=4$ higher spin algebra for low spins
and present how to determine the structure constants, which depend on the
higher spin algebra parameter, in general, for fixed spins $s_1$ and $s_2$.
