  Performance is an important non-functional aspect of the software
requirement. Modern software systems are highly-configurable and
misconfigurations may easily cause performance issues. A software system that
suffers performance issues may exhibit low program throughput and long response
time. However, the sheer size of the configuration space makes it challenging
for administrators to manually select and adjust the configuration options to
achieve better performance. In this paper, we propose ConfRL, an approach to
tune software performance automatically. The key idea of ConfRL is to use
reinforcement learning to explore the configuration space by a trial-and-error
approach and to use the feedback received from the environment to tune
configuration option values to achieve better performance. To reduce the cost
of reinforcement learning, ConfRL employs sampling, clustering, and dynamic
state reduction techniques to keep states in a large configuration space
manageable. Our evaluation of four real-world highly-configurable server
programs shows that ConfRL can efficiently and effectively guide software
systems to achieve higher long-term performance.
