  Traditional data mining algorithms are exceptional at seeing patterns in data
that humans cannot, but are often confused by details that are obvious to the
organic eye. Algorithms that include humans "in-the-loop" have proved
beneficial for accuracy by allowing a user to provide direction in these
situations, but the slowness of human interactions causes execution times to
increase exponentially. Thus, we seek to formalize frameworks that include
humans "over-the-loop", giving the user an option to intervene when they deem
it necessary while not having user feedback be an execution requirement. With
this strategy, we hope to increase the accuracy of solutions with minimal
losses in execution time. This paper describes our vision of this strategy and
associated problems.
