  A joint degree matrix (JDM) specifies the number of connections between nodes
of given degrees in a graph, for all degree pairs and uniquely determines the
degree sequence of the graph. We consider the space of all balanced
realizations of an arbitrary JDM, realizations in which the links between any
two degree groups are placed as uniformly as possible. We prove that a swap
Markov Chain Monte Carlo (MCMC) algorithm in the space of all balanced
realizations of an {\em arbitrary} graphical JDM mixes rapidly, i.e., the
relaxation time of the chain is bounded from above by a polynomial in the
number of nodes $n$. To prove fast mixing, we first prove a general
factorization theorem similar to the Martin-Randall method for disjoint
decompositions (partitions). This theorem can be used to bound from below the
spectral gap with the help of fast mixing subchains within every partition and
a bound on an auxiliary Markov chain between the partitions. Our proof of the
general factorization theorem is direct and uses conductance based methods
(Cheeger inequality).
