  Large Hubbard U limit of the Kane-Mele model on a zigzag ribbon of honeycomb
lattice near half-filling is studied via a renormalized mean-field theory. The
ground state exhibits time-reversal symmetry (TRS) breaking d + i d'-wave
superconductivity. At large spin-orbit coupling, the Z2 phase with non-trivial
spin Chern number in the pure Kane-Mele model is persistent into the TRS broken
state (called spin-Chern phase), and has two pairs of counter-propagating
helical Majorana modes at the edges. As the spin-orbit coupling is reduced, the
system undergoes a topological quantum phase transition from the spin-Chern to
chiral superconducting states. Possible relevance of our results to
adatom-doped graphene and irridate compounds is discussed.
