  Integrated Sensing and Communication (ISAC) has attracted substantial
attraction in recent years for spectral efficiency improvement, enabling
hardware and spectrum sharing for simultaneous sensing and signaling
operations. In-band Full Duplex (FD) is being considered as a key enabling
technology for ISAC applications due to its simultaneous transmission and
reception capability. In this paper, we present an FD-based ISAC system
operating at millimeter Wave (mmWave) frequencies, where a massive
Multiple-Input Multiple-Output (MIMO) Base Station (BS) node employing hybrid
Analog and Digital (A/D) beamforming is communicating with a DownLink (DL)
multi-antenna user and the same waveform is utilized at the BS receiver for
sensing the radar targets in its coverage environment. We develop a sensing
algorithm that is capable of estimating Direction of Arrival (DoA), range, and
relative velocity of the radar targets. A joint optimization framework for
designing the A/D transmit and receive beamformers as well as the
Self-Interference (SI) cancellation is presented with the objective to maximize
the achievable DL rate and the accuracy of the radar target sensing
performance. Our simulation results, considering fifth Generation (5G)
Orthogonal Frequency Division Multiplexing (OFDM) waveforms, verify our
approach's high precision in estimating DoA, range, and velocity of multiple
radar targets, while maximizing the DL communication rate.
