  Existing solutions to network scheduling typically assume that the
instantaneous link rates are completely known before a scheduling decision is
made or consider a bandit setting where the accurate link quality is discovered
only after it has been used for data transmission. In practice, the decision
maker can obtain (relatively accurate) channel information, e.g., through
beamforming in mmWave networks, right before data transmission. However,
frequent beamforming incurs a formidable overhead in densely deployed mmWave
WLANs. In this paper, we consider the important problem of throughput
optimization with joint link probing and scheduling. The problem is challenging
even when the link rate distributions are pre-known (the offline setting) due
to the necessity of balancing the information gains from probing and the cost
of reducing the data transmission opportunity. We develop an approximation
algorithm with guaranteed performance when the probing decision is
non-adaptive, and a dynamic programming based solution for the more challenging
adaptive setting. We further extend our solutions to the online setting with
unknown link rate distributions and develop a contextual-bandit based algorithm
and derive its regret bound. Numerical results using data traces collected from
real-world mmWave deployments demonstrate the efficiency of our solutions.
