  In this paper, we consider the precoder design for an underloaded or
critically loaded downlink multi-user multiple-input multiple-output (MIMO)
communication system. We propose novel precoding and decoding schemes which
enhance system performance based on rate splitting at the transmitter and
single-stage successive interference cancellation at the receivers. The
proposed successive null-space (SNS) precoding utilizes linear combinations of
the null-space basis vectors of the successively augmented MIMO channel
matrices of the users as precoding vectors to adjust the
inter-user-interference experienced by the receivers. We formulate a non-convex
weighted sum rate optimization problem for the precoding vectors and the
associated power allocation for the proposed SNS-based MIMO-rate-splitting
multiple access (RSMA) scheme. We obtain a suboptimal solution for this problem
via successive convex approximation. Moreover, we study the robustness of the
proposed precoding scheme to imperfect channel state information (CSI) at the
base station via derivative-based sensitivity analysis. Our analysis and
simulation results reveal the enhanced performance and robustness of the
proposed SNS-based MIMO-RSMA scheme over several baseline multi-user MIMO
schemes, especially for imperfect CSI.
