  We investigate the problem of multi-hop scheduling in self-backhauled
millimeter wave (mmWave) networks. Owing to the high path loss and blockage of
mmWave links, multi-hop paths between the macro base station and the intended
users via full-duplex small cells need to be carefully selected. This paper
addresses the fundamental question: how to select the best paths and how to
allocate rates over these paths subject to latency constraints. To answer this
question, we propose a new system design, which factors in mmWave-specific
channel variations and network dynamics. The problem is cast as a network
utility maximization subject to a bounded delay constraint and network
stability. The studied problem is decoupled into: (i) a path selection and (ii)
rate allocation, whereby learning the best paths is done by means of a
reinforcement learning algorithm, and the rate allocation is solved by applying
the successive convex approximation method. Via numerical results, our approach
ensures reliable communication with a guaranteed probability of 99.9999%, and
reduces latency by 50.64% and 92.9% as compared to baselines.
