  Coset constructions of $\mathcal{W}$-algebras have many applications, and
were recently given for principal $\mathcal{W}$-algebras of $A$, $D$, and $E$
types by Arakawa together with the first and third authors. In this paper, we
give coset constructions of the large and small $N=4$ superconformal algebras,
which are the minimal $\mathcal{W}$-algebras of $\mathfrak{d}(2,1;a)$ and
$\mathfrak{psl}(2|2)$, respectively. From these realizations, one finds a
remarkable connection between the large $N=4$ algebra and the diagonal coset
$C^{k_1, k_2} = \text{Com}(V^{k_1+k_2}(\mathfrak{sl}_2),
V^{k_1}(\mathfrak{sl}_2) \otimes V^{k_2}(\mathfrak{sl}_2))$, namely, as
two-parameter vertex algebras, $C^{k_1, k_2}$ coincides with the coset of the
large $N=4$ algebra by its affine subalgebra. We also show that at special
points in the parameter space, the simple quotients of these cosets are
isomorphic to various $\mathcal{W}$-algebras. As a corollary, we give new
examples of strongly rational principal $\mathcal{W}$-algebras of type $C$ at
degenerate admissible levels.
