  Since the signal with strong power should be demodulated first for successive
interference cancellation (SIC) demodulation in non-orthogonal multiple access
(NOMA) systems, the base station (BS) should inform the near user terminal
(UT), which has allocated higher power, of modulation mode of the far user
terminal. To avoid unnecessary signaling overhead in this process, a blind
detection algorithm of NOMA signal modulation mode is designed in this paper.
Taking the joint constellation density diagrams of NOMA signal as the detection
features, deep residual network is built for classification, so as to detect
the modulation mode of NOMA signal. In view of the fact that the joint
constellation diagrams are easily polluted by high intensity noise and lose
their real distribution pattern, the wavelet denoising method is adopted to
improve the quality of constellations. The simulation results represent that
the proposed algorithm can achieve satisfactory detection accuracy in NOMA
systems. In addition, the factors affecting the recognition performance are
also verified and analyzed.
