  In this paper, we propose to use hybrid relay-intelligent reflecting surface
(HR-IRS) to improve the security performance of directional modulation (DM)
system. In particular, the eavesdropper in this system works in full-duplex
(FD) mode and he will eavesdrop on the confidential message (CM) as well as
send malicious jamming. We aim to maximize the secrecy rate (SR) by jointly
optimizing the receive beamforming, transmit beamforming and phase shift matrix
(PSM) of HR-IRS. Since the optimization problem is un-convex and the variables
are coupled to each other, we solve this problem by iteratively optimizing
these variables. The receive beamforming and transmit beamforming are obtained
based on generalized Rayleigh-Ritz theorem and Dinkelbach's Transform
respectively. And for PSM, two methods, called separate optimization of PSM
(SO-PSM) and joint optimization of PSM (JO-PSM) are proposed. Thus, two
iterative algorithms are proposed accordingly, namely maximizing SR based on
SO-PSM (Max-SR-SOP) and maximizing SR based on JO-PSM (Max-SR-JOP). The former
has better performance and the latter has lower complexity. The simulation
results show that when HR-IRS has sufficient power budget, the proposed
Max-SR-SOP and Max-SR-JOP can enable HR-IRS-aided DM network to obtain higher
SR than passive IRS-aided DM network.
