  We consider a transmitter with mmWave/sub6 dual interfaces. Due to the
intermittency of mmWave channel, the transmitter must schedule packets wisely
across the interfaces to minimize the average delay by observing the system
state. We usethe well-known dynamic programming methods and Q-learning to find
the optimal scheduling policy and investigate the influenceof observing CSI on
the optimal policy under different levels of knowledge of the environment. We
find that only when the channel state transition model is not available, the
instantaneousCSI can help in reducing system delay
