  In this paper, we investigate the hybrid beamforming design for a
multiple-input multiple-output (MIMO) integrated sensing and communication
(ISAC) system, where a multi-antenna base station (BS) with hybrid
analog-digital transmit antenna arrays sends dual-functional signals to
communicate with a multi-antenna user and simultaneously sense the location
information of a point target based on the reflected echo signals.
Specifically, we aim to sense the target's unknown and random angle information
by exploiting its prior distribution information, with posterior Cram\'{e}r-Rao
bound (PCRB) employed as the sensing performance metric. First, we consider a
sensing-only case and study the hybrid beamforming optimization to minimize the
sensing PCRB. We analytically prove that hybrid beamforming can achieve the
same performance as the optimized digital beamforming as long as the number of
radio frequency (RF) chains is larger than 1. Then, we propose a convex
relaxation based algorithm for the hybrid beamforming design with a single RF
chain. Next, we study the hybrid beamforming optimization to minimize the PCRB
subject to a communication rate target. Due to the intractability of the exact
PCRB expression, we replace it with a tight upper bound. Although this problem
is still non-convex and challenging to solve, we propose an alternating
optimization (AO) algorithm for finding a high-quality suboptimal solution
based on the feasible point pursuit successive convex approximation (FPP-SCA)
method. Numerical results validate the effectiveness of our proposed hybrid
beamforming design.
