  In this paper we study the algebra of quantum observables of the Chern-Simons
matrix model which was originally proposed by Susskind and Polychronakos to
describe electrons in fractional quantum Hall effects. We establish the
commutation relations for its generators and study the large $N$ limit of its
representation. We show that the large $N$ limit algebra is isomorphic to the
uniform in $N$ algebra studied by Costello, which is isomorphic to the deformed
double current algebra studied by Guay. Under appropriate scaling limit, we
show that the large $N$ limit algebra degenerates to a Lie algebra which admits
a surjective map to the affine Lie algebra of $\mathfrak{u}(p)$. This leads to
a complete proof of the large $N$ emergence of the $\mathfrak{u}(p)$ current
algebra as proposed by Dorey, Tong and Turner. This also suggests a rigorous
derivation of edge excitation of a fractional quantum Hall droplet.
