  Relevant and high-quality data are critical to successful development of
machine learning applications. For machine learning applications on dynamic
systems equipped with a large number of sensors, such as connected vehicles and
robots, how to find relevant and high-quality data features in an efficient way
is a challenging problem. In this work, we address the problem of feature
selection in constrained continuous data acquisition. We propose a
feedback-based dynamic feature selection algorithm that efficiently decides on
the feature set for data collection from a dynamic system in a step-wise
manner. We formulate the sequential feature selection procedure as a Markov
Decision Process. The machine learning model performance feedback with an
exploration component is used as the reward function in an $\epsilon$-greedy
action selection. Our evaluation shows that the proposed feedback-based feature
selection algorithm has superior performance over constrained baseline methods
and matching performance with unconstrained baseline methods.
