  Learning with streaming data has attracted much attention during the past few
years. Though most studies consider data stream with fixed features, in real
practice the features may be evolvable. For example, features of data gathered
by limited-lifespan sensors will change when these sensors are substituted by
new ones. In this paper, we propose a novel learning paradigm: \emph{Feature
Evolvable Streaming Learning} where old features would vanish and new features
would occur. Rather than relying on only the current features, we attempt to
recover the vanished features and exploit it to improve performance.
Specifically, we learn two models from the recovered features and the current
features, respectively. To benefit from the recovered features, we develop two
ensemble methods. In the first method, we combine the predictions from two
models and theoretically show that with the assistance of old features, the
performance on new features can be improved. In the second approach, we
dynamically select the best single prediction and establish a better
performance guarantee when the best model switches. Experiments on both
synthetic and real data validate the effectiveness of our proposal.
