  Business analytics and machine learning have become essential success factors
for various industries - with the downside of cost-intensive gathering and
labeling of data. Few-shot learning addresses this challenge and reduces data
gathering and labeling costs by learning novel classes with very few labeled
data. In this paper, we design a human-in-the-loop (HITL) system for few-shot
learning and analyze an extensive range of mechanisms that can be used to
acquire human expert knowledge for instances that have an uncertain prediction
outcome. We show that the acquisition of human expert knowledge significantly
accelerates the few-shot model performance given a negligible labeling effort.
We validate our findings in various experiments on a benchmark dataset in
computer vision and real-world datasets. We further demonstrate the
cost-effectiveness of HITL systems for few-shot learning. Overall, our work
aims at supporting researchers and practitioners in effectively adapting
machine learning models to novel classes at reduced costs.
