  To improve the accuracy of direction-of-arrival (DOA) estimation, a deep
learning (DL)-based method called CDAE-DNN is proposed for hybrid analog and
digital (HAD) massive MIMO receive array with overlapped subarray (OSA)
architecture in this paper. In the proposed method, the sample covariance
matrix (SCM) is first input to a convolution denoise autoencoder (CDAE) to
remove the approximation error, then the output of CDAE is imported to a
fully-connected (FC) network to get the estimation result. Based on the
simulation results, the proposed CDAE-DNN has great performance advantages over
traditional MUSIC algorithm and CNN-based method, especially in the situations
with low signal to noise ratio (SNR) and low snapshot numbers. And the OSA
architecture has also been shown to significantly improve the estimation
accuracy compared to non-overlapped subarray (NOSA) architecture. In addition,
the Cramer-Rao lower bound (CRLB) for the HAD-OSA architecture is presented.
