  Applications such as virtual reality and online gaming require low delays for
acceptable user experience. A key task for over-the-top (OTT) service providers
who provide these applications is sending traffic through the networks to
minimize delays. OTT traffic is typically generated from multiple data centers
which are multi-homed to several network ingresses. However, information about
the path characteristics of the underlying network from the ingresses to
destinations is not explicitly available to OTT services. These can only be
inferred from external probing. In this paper, we combine network tomography
with machine learning to minimize delays. We consider this problem in a general
setting where traffic sources can choose a set of ingresses through which their
traffic enter a black box network. The problem in this setting can be viewed as
a reinforcement learning problem with constraints on a continuous action space,
which to the best of our knowledge have not been investigated by the machine
learning community. Key technical challenges to solving this problem include
the high dimensionality of the problem and handling constraints that are
intrinsic to networks. Evaluation results show that our methods achieve up to
60% delay reductions in comparison to standard heuristics. Moreover, the
methods we develop can be used in a centralized manner or in a distributed
manner by multiple independent agents.
