  In this paper, we propose a novel non-orthogonal multiple access (NOMA)
scheme based on trellis-coded modulation (TCM). Different from those in the
traditional code-domain NOMA, the incoming data streams of multiple users are
jointly coded and mapped to the codewords so as to improve the coding gain of
the system without any bandwidth extension. Both codeword design and multi-user
detection are considered to optimize the TCM for NOMA. The multi-dimensional
(MD) constellation is carefully selected and a new set partitioning algorithm
is developed for MD signal set labeling. To achieve the trade-off between the
BER performance and complexity, a suboptimal two-layer Viterbi algorithm is
proposed for joint decoding. Simulation results show that our proposed
TCM-based NOMA scheme performs significantly better than the traditional
code-domain NOMA and OFDMA schemes in terms of the BER performance under the
same channel conditions.
