  Topology impacts important network performance metrics, including link
utilization, throughput and latency, and is of central importance to network
operators. However, due to the combinatorial nature of network topology, it is
extremely difficult to obtain an optimal solution, especially since topology
planning in networks also often comes with management-specific constraints. As
a result, local optimization with hand-tuned heuristic methods from human
experts are often adopted in practice. Yet, heuristic methods cannot cover the
global topology design space while taking into account constraints, and cannot
guarantee to find good solutions.
  In this paper, we propose a novel deep reinforcement learning (DRL)
algorithm, called Advantage Actor Critic-Graph Searching (A2C-GS), for network
topology optimization. A2C-GS consists of three novel components, including a
verifier to validate the correctness of a generated network topology, a graph
neural network (GNN) to efficiently approximate topology rating, and a DRL
actor layer to conduct a topology search. A2C-GS can efficiently search over
large topology space and output topology with satisfying performance. We
conduct a case study based on a real network scenario, and our experimental
results demonstrate the superior performance of A2C-GS in terms of both
efficiency and performance.
