  Involving users in early phases of software development has become a common
strategy as it enables developers to consider user needs from the beginning.
Once a system is in production, new opportunities to observe, evaluate and
learn from users emerge as more information becomes available. Gathering
information from users to continuously evaluate their behavior is a common
practice for commercial software, while the Cranfield paradigm remains the
preferred option for Information Retrieval (IR) and recommendation systems in
the academic world. Here we introduce the Infrastructures for Living Labs
STELLA project which aims to create an evaluation infrastructure allowing
experimental systems to run along production web-based academic search systems
with real users. STELLA combines user interactions and log files analyses to
enable large-scale A/B experiments for academic search.
