  The topological color code and the toric code are two leading candidates for
realizing fault-tolerant quantum computation. Here we show that the color code
on a $d$-dimensional closed manifold is equivalent to multiple decoupled copies
of the $d$-dimensional toric code up to local unitary transformations and
adding or removing ancilla qubits. Our result not only generalizes the proven
equivalence for $d=2$, but also provides an explicit recipe of how to decouple
independent components of the color code, highlighting the importance of
colorability in the construction of the code. Moreover, for the $d$-dimensional
color code with $d+1$ boundaries of $d+1$ distinct colors, we find that the
code is equivalent to multiple copies of the $d$-dimensional toric code which
are attached along a $(d-1)$-dimensional boundary. In particular, for $d=2$, we
show that the (triangular) color code with boundaries is equivalent to the
(folded) toric code with boundaries. We also find that the $d$-dimensional
toric code admits logical non-Pauli gates from the $d$-th level of the Clifford
hierarchy, and thus saturates the bound by Bravyi and K\"{o}nig. In particular,
we show that the $d$-qubit control-$Z$ logical gate can be fault-tolerantly
implemented on the stack of $d$ copies of the toric code by a local unitary
transformation.
