  This paper proposes a novel neural network architecture, that we call an
auto-precoder, and a deep-learning based approach that jointly senses the
millimeter wave (mmWave) channel and designs the hybrid precoding matrices with
only a few training pilots. More specifically, the proposed machine learning
model leverages the prior observations of the channel to achieve two
objectives. First, it optimizes the compressive channel sensing vectors based
on the surrounding environment in an unsupervised manner to focus the sensing
power on the most promising spatial directions. This is enabled by a novel
neural network architecture that accounts for the constraints on the RF chains
and models the transmitter/receiver measurement matrices as two complex-valued
convolutional layers. Second, the proposed model learns how to construct the RF
beamforming vectors of the hybrid architectures directly from the projected
channel vector (the received sensing vector). The auto-precoder neural network
that incorporates both the channel sensing and beam prediction is trained
end-to-end as a multi-task classification problem. Thanks to this design
methodology that leverages the prior channel observations and the implicit
awareness about the surrounding environment/user distributions, the proposed
approach significantly reduces the training overhead compared to classical
(non-machine learning) solutions. For example, for a system of 64 transmit and
64 receive antennas, with 3 RF chains at both sides, the proposed solution
needs only 8 or 16 channel training pilots to directly predict the RF
beamforming/combining vectors of the hybrid architectures and achieve
near-optimal achievable rates. This highlights a promising solution for the
channel estimation and hybrid precoding design problem in mmWave and massive
MIMO systems.
