  Traditional Traffic Engineering (TE) solutions can achieve the optimal or
near-optimal performance by rerouting as many flows as possible. However, they
do not usually consider the negative impact, such as packet out of order, when
frequently rerouting flows in the network. To mitigate the impact of network
disturbance, one promising TE solution is forwarding the majority of traffic
flows using Equal-Cost Multi-Path (ECMP) and selectively rerouting a few
critical flows using Software-Defined Networking (SDN) to balance link
utilization of the network. However, critical flow rerouting is not trivial
because the solution space for critical flow selection is enormous. Moreover,
it is impossible to design a heuristic algorithm for this problem based on
fixed and simple rules, since rule-based heuristics are unable to adapt to the
changes of the traffic matrix and network dynamics. In this paper, we propose
CFR-RL (Critical Flow Rerouting-Reinforcement Learning), a Reinforcement
Learning-based scheme that learns a policy to select critical flows for each
given traffic matrix automatically. CFR-RL then reroutes these selected
critical flows to balance link utilization of the network by formulating and
solving a simple Linear Programming (LP) problem. Extensive evaluations show
that CFR-RL achieves near-optimal performance by rerouting only 10%-21.3% of
total traffic.
