  Traditional multicast routing methods have some problems in constructing a
multicast tree, such as limited access to network state information, poor
adaptability to dynamic and complex changes in the network, and inflexible data
forwarding. To address these defects, the optimal multicast routing problem in
software-defined networking (SDN) is tailored as a multi-objective optimization
problem, and an intelligent multicast routing algorithm DRL-M4MR based on the
deep Q network (DQN) deep reinforcement learning (DRL) method is designed to
construct a multicast tree in SDN. First, the multicast tree state matrix, link
bandwidth matrix, link delay matrix, and link packet loss rate matrix are
designed as the state space of the DRL agent by combining the global view and
control of the SDN. Second, the action space of the agent is all the links in
the network, and the action selection strategy is designed to add the links to
the current multicast tree under four cases. Third, single-step and final
reward function forms are designed to guide the intelligence to make decisions
to construct the optimal multicast tree. The experimental results show that,
compared with existing algorithms, the multicast tree construct by DRL-M4MR can
obtain better bandwidth, delay, and packet loss rate performance after
training, and it can make more intelligent multicast routing decisions in a
dynamic network environment.
