  Cooperative transmission is an emerging communication technique that takes
advantage of the broadcast nature of wireless channels. However, due to low
spectral efficiency and the requirement of orthogonal channels, its potential
for use in future wireless networks is limited. In this paper, by making use of
multiuser detection (MUD) and network coding, cooperative transmission
protocols with high spectral efficiency, diversity order, and coding gain are
developed. Compared with the traditional cooperative transmission protocols
with single-user detection, in which the diversity gain is only for one source
user, the proposed MUD cooperative transmission protocols have the merit that
the improvement of one user's link can also benefit the other users. In
addition, using MUD at the relay provides an environment in which network
coding can be employed. The coding gain and high diversity order can be
obtained by fully utilizing the link between the relay and the destination.
  From the analysis and simulation results, it is seen that the proposed
protocols achieve higher diversity gain, better asymptotic efficiency, and
lower bit error rate, compared to traditional MUD schemes and to existing
cooperative transmission protocols. From the simulation results, the
performance of the proposed scheme is near optimal as the performance gap is
0.12dB for average bit error rate (BER) 10^{-6} and 1.04dB for average BER
10^(-3), compared to two performance upper bounds.
