  This paper considers the joint antenna selection (AS) problem for a classical
two-user MIMO non-orthogonal multiple access (NOMA) system, where both the base
station (BS) and users (UEs) are equipped with multiple antennas. Specifically,
several computationally-efficient AS algorithms are developed for two
commonly-used NOMA scenarios: fixed power allocation NOMA (F-NOMA) and
cognitive radio-inspired NOMA (CR-NOMA). For the F-NOMA system, two novel AS
schemes, namely max-max-max AS (A$^3$-AS) and max-min-max AS (AIA-AS), are
proposed to maximize the system sum-rate, without and with the consideration of
user fairness, respectively. In the CR-NOMA network, a novel AS algorithm,
termed maximum-channel-gain-based AS (MCG-AS), is proposed to maximize the
achievable rate of the secondary user, under the condition that the primary
user's quality of service requirement is satisfied. The asymptotic closed-form
expressions of the average sum-rate for A$^3$-AS and AIA-AS and that of the
average rate of the secondary user for MCG-AS are derived, respectively.
Numerical results demonstrate that the AIA-AS provides better user-fairness,
while the A$^3$-AS achieves a near-optimal sum-rate in F-NOMA systems. For the
CR-NOMA scenario, MCG-AS achieves a near-optimal performance in a wide SNR
regime. Furthermore, all the proposed AS algorithms yield a significant
computational complexity reduction, compared to exhaustive search-based
counterparts.
