  It is well known that the quantum double structure plays an important role in
three dimensional quantum gravity coupled to matter field. In this paper, we
show how this algebraic structure emerges in the context of three dimensional
Riemannian loop quantum gravity (LQG) coupled to a finite number of massive
spinless point particles. In LQG, physical states are usually constructed from
the notion of SU(2) cylindrical functions on a Riemann surface $\Sigma$ and the
Hilbert structure is defined by the Ashtekar-Lewandowski measure. In the case
where $\Sigma$ is the sphere $S^2$, we show that the physical Hilbert space is
in fact isomorphic to a tensor product of simple unitary representations of the
Drinfeld double DSU(2): the masses of the particles label the simple
representations, the physical states are tensor products of vectors of simple
representations and the physical scalar product is given by intertwining
coefficients between simple representations. This result is generalized to the
case of any Riemann surface $\Sigma$.
