  To evaluate software maintenance techniques and tools in controlled
experiments with human participants, researchers currently use projects and
tasks selected on an ad-hoc basis. This can unrealistically favor their tool,
and it makes the comparison of results difficult. We suggest a gradual creation
of a benchmark repository with projects, tasks, and metadata relevant for
human-based studies. In this paper, we discuss the requirements and challenges
of such a repository, along with the steps which could lead to its
construction.
