  We formulate a stability conjecture for the coefficients of the colored Jones
polynomial of a knot, colored by irreducible representations in a fixed ray of
a simple Lie algebra, and verify it for all torus knots and all simple Lie
algebras of rank $2$. Our conjecture is motivated by a structure theorem for
the degree and the coefficients of a $q$-holonomic sequence of polynomials
given in [Ga2] and by a stability theorem of the colored Jones polynomial of an
alternating knot given in \cite{GL2}. We illustrate our results with sample
computations.
