  The innovative services empowered by the Internet of Things (IoT) require a
seamless and reliable wireless infrastructure that enables communications
within heterogeneous and dynamic low-power and lossy networks (LLNs). The
Routing Protocol for LLNs (RPL) was designed to meet the communication
requirements of a wide range of IoT application domains. However, a load
balancing problem exists in RPL under heavy traffic-load scenarios, degrading
the network performance in terms of delay and packet delivery. In this paper,
we tackle the problem of load-balancing in RPL networks using a
reinforcement-learning framework. The proposed method adopts Q-learning at each
node to learn an optimal parent selection policy based on the dynamic network
conditions. Each node maintains the routing information of its neighbours as
Q-values that represent a composite routing cost as a function of the
congestion level, the link-quality and the hop-distance. The Q-values are
updated continuously exploiting the existing RPL signalling mechanism. The
performance of the proposed approach is evaluated through extensive simulations
and compared with the existing work to demonstrate its effectiveness. The
results show that the proposed method substantially improves network
performance in terms of packet delivery and average delay with a marginal
increase in the signalling frequency.
