  Wireless sensor networks (WSNs) are the foundation of the Internet of Things
(IoT), and in the era of the fifth generation of wireless communication
networks, they are envisioned to be truly ubiquitous, reliable, scalable, and
energy efficient. To this end, topology control is an important mechanism to
realize self-organized WSNs that are capable of adapting to the dynamics of the
environment. Topology optimization is combinatorial in nature, and generally is
NP-hard to solve. Most existing algorithms leverage heuristic rules to reduce
the number of search candidates so as to obtain a suboptimal solution in a
certain sense. In this paper, we propose a deep reinforcement learning-based
topology optimization algorithm, a unified search framework, for self-organized
energy-efficient WSNs. Specifically, the proposed algorithm uses a deep neural
network to guide a Monte Carlo tree search to roll out simulations, and the
results from the tree search reinforce the learning of the neural network. In
addition, the proposed algorithm is an anytime algorithm that keeps improving
the solution with an increasing amount of computing resources. Various
simulations show that the proposed algorithm achieves better performance as
compared to heuristic solutions, and is capable of adapting to environment and
network changes without restarting the algorithm from scratch.
