  Interactive user interfaces need to continuously evolve based on the
interactions that a user has (or does not have) with the system. This may
require constant exploration of various options that the system may have for
the user and obtaining signals of user preferences on those. However, such an
exploration, especially when the set of available options itself can change
frequently, can lead to sub-optimal user experiences. We present
Explore-Exploit: a framework designed to collect and utilize user feedback in
an interactive and online setting that minimizes regressions in end-user
experience. This framework provides a suite of online learning operators for
various tasks such as personalization ranking, candidate selection and active
learning. We demonstrate how to integrate this framework with run-time services
to leverage online and interactive machine learning out-of-the-box. We also
present results demonstrating the efficiencies that can be achieved using the
Explore-Exploit framework.
