  During network planning phase, optimal network planning implemented through
efficient resource allocation and static traffic demand provisioning in
IP-over-elastic optical network (IP-over-EON) is significantly challenging
compared with the fixed-grid wavelength division multiplexing (WDM) network due
to increased flexibility in IP-over-EON. Mathematical optimization models used
for this purpose may not provide solution for large networks due to large
computational complexity. In this regard, a greedy heuristic may be used that
intuitively selects traffic elements in sequence from static traffic demand
matrix and attempts to find the best solution. However, in general, such greedy
heuristics offer suboptimal solutions, since appropriate traffic sequence
offering the optimal performance is rarely selected. In this regard, we propose
a reinforcement learning technique (in particular a Q-learning method),
combined with an auxiliary graph (AG)-based energy efficient greedy method to
be used for large network planning. The Q-learning method is used to decide the
suitable sequence of traffic allocation such that the overall power consumption
in the network reduces. In the proposed heuristic, each traffic from the given
static traffic demand matrix is successively selected using the Q-learning
technique and provisioned using the AG-based greedy method.
