  Recent networking research has identified that data-driven congestion control
(CC) can be more efficient than traditional CC in TCP. Deep reinforcement
learning (RL), in particular, has the potential to learn optimal network
policies. However, RL suffers from instability and over-fitting, deficiencies
which so far render it unacceptable for use in datacenter networks. In this
paper, we analyze the requirements for RL to succeed in the datacenter context.
We present a new emulator, Iroko, which we developed to support different
network topologies, congestion control algorithms, and deployment scenarios.
Iroko interfaces with the OpenAI gym toolkit, which allows for fast and fair
evaluation of different RL and traditional CC algorithms under the same
conditions. We present initial benchmarks on three deep RL algorithms compared
to TCP New Vegas and DCTCP. Our results show that these algorithms are able to
learn a CC policy which exceeds the performance of TCP New Vegas on a dumbbell
and fat-tree topology. We make our emulator open-source and publicly available:
https://github.com/dcgym/iroko
