  One prerequisite for supervised machine learning is high quality labelled
data. Acquiring such data is, particularly if expert knowledge is required,
costly or even impossible if the task needs to be performed by a single expert.
In this paper, we illustrate tool support that we adopted and extended to
source domain-specific knowledge from experts. We provide insight in design
decisions that aim at motivating experts to dedicate their time at performing
the labelling task. We are currently using the approach to identify true
synonyms from a list of candidate synonyms. The identification of synonyms is
important in scenarios were stakeholders from different companies and
background need to collaborate, for example when defining and negotiating
requirements. We foresee that the approach of expert-sourcing is applicable to
any data labelling task in software engineering. The discussed design decisions
and implementation are an initial draft that can be extended, refined and
validated with further application.
