  In this paper, an unsupervised deep learning framework based on dual-path
model-driven variational auto-encoders (VAE) is proposed for angle-of-arrivals
(AoAs) and channel estimation in massive MIMO systems. Specifically designed
for channel estimation, the proposed VAE differs from the original VAE in two
aspects. First, the encoder is a dual-path neural network, where one path uses
the received signal to estimate the path gains and path angles, and another
uses the correlation matrix of the received signal to estimate AoAs. Second,
the decoder has fixed weights that implement the signal propagation model,
instead of learnable parameters. This knowledge-aware decoder forces the
encoder to output meaningful physical parameters of interests (i.e., path
gains, path angles, and AoAs), which cannot be achieved by original VAE.
Rigorous analysis is carried out to characterize the multiple global optima and
local optima of the estimation problem, which motivates the design of the
dual-path encoder. By alternating between the estimation of path gains, path
angles and the estimation of AoAs, the encoder is proved to converge. To
further improve the convergence performance, a low-complexity procedure is
proposed to find good initial points. Numerical results validate theoretical
analysis and demonstrate the performance improvements of our proposed
framework.
