  We study a large-N generalization of $J_1$-$J_2$ Heisenberg model on square
lattice -- an $Sp(2N)$ spin model. The possible quantum spin liquid phases of
the $Sp(2N)$ model are studied using the SU(2) projective construction. We find
several spin liquid states at least in large $N$ limit, which include SU(2)
$\pi$-flux state, SU(2) chiral spin state and $Z_2$ spin liquid states. All
those spin liquid states have non-trivial quantum orders. We show how
projective symmetry group, which characterizes quantum order, protects the
stability of even gapless spin liquids. We also study the continuous quantum
phase transition from the SU(2) $\pi$-flux state to the SU(2) chiral spin
liquid state, and from the SU(2) $\pi$-flux state to the $Z_2$ spin liquid
state. We show that those phase transitions are beyond the paradigm of Landau
symmetry-breaking theory. The first phase transition, although a $Z_2$ symmetry
breaking transition, contains critical exponents that are different from those
obtained from the Ginzburg-Landau theory of Ising universality class. The
second transition does not even involve a change of symmetry and has no
symmetry breaking order parameter.
