  The existing network attack and defense method can be regarded as game, but
most of the game only involves network domain, not multiple domain cyberspace.
To address this challenge, this paper proposed a multiple domain cyberspace
attack and defense game model based on reinforcement learning. We define the
multiple domain cyberspace include physical domain, network domain and digital
domain. By establishing two agents, representing the attacker and the defender
respectively, defender will select the multiple domain actions in the multiple
domain cyberspace to obtain defender's optimal reward by reinforcement
learning. In order to improve the defense ability of defender, a game model
based on reward randomization reinforcement learning is proposed. When the
defender takes the multiple domain defense action, the reward is randomly given
and subject to linear distribution, so as to find the better defense policy and
improve defense success rate. The experimental results show that the game model
can effectively simulate the attack and defense state of multiple domain
cyberspace, and the proposed method has a higher defense success rate than DDPG
and DQN.
