  In this paper, we propose a new nonlinear detector with improved interference
suppression in Multi-User Multiple Input, Multiple Output (MU-MIMO) system. The
proposed detector is a combination of the following parts: QR decomposition
(QRD), low complexity users sorting before QRD, sorting-reduced (SR) K-best
method and minimum mean square error (MMSE) pre-processing. Our method
outperforms a linear interference rejection combining (IRC, i.e. MMSE
naturally) method significantly in both strong interference and additive white
noise scenarios with both ideal and real channel estimations. This result has
wide application importance for scenarios with strong interference, i.e. when
co-located users utilize the internet in stadium, highway, shopping center,
etc. Simulation results are presented for the non-line of sight 3D-UMa model of
5G QuaDRiGa 2.0 channel for 16 highly correlated single-antenna users with
QAM16 modulation in 64 antennas of Massive MIMO system. The performance was
compared with MMSE and other detection approaches.
