  A self-adaptive system can modify its own structure and behavior at runtime
based on its perception of the environment, of itself and of its requirements.
To develop a self-adaptive system, software developers codify knowledge about
the system and its environment, as well as how adaptation actions impact on the
system. However, the codified knowledge may be insufficient due to design time
uncertainty, and thus a self-adaptive system may execute adaptation actions
that do not have the desired effect. Online learning is an emerging approach to
address design time uncertainty by employing machine learning at runtime.
Online learning accumulates knowledge at runtime by, for instance, exploring
not-yet executed adaptation actions. We address two specific problems with
respect to online learning for self-adaptive systems. First, the number of
possible adaptation actions can be very large. Existing online learning
techniques randomly explore the possible adaptation actions, but this can lead
to slow convergence of the learning process. Second, the possible adaptation
actions can change as a result of system evolution. Existing online learning
techniques are unaware of these changes and thus do not explore new adaptation
actions, but explore adaptation actions that are no longer valid. We propose
using feature models to give structure to the set of adaptation actions and
thereby guide the exploration process during online learning. Experimental
results involving four real-world systems suggest that considering the
hierarchical structure of feature models may speed up convergence by 7.2% on
average. Considering the differences between feature models before and after an
evolution step may speed up convergence by 64.6% on average. [...]
