  Learning visual concepts from raw images without strong supervision is a
challenging task. In this work, we show the advantages of prototype
representations for understanding and revising the latent space of neural
concept learners. For this purpose, we introduce interactive Concept Swapping
Networks (iCSNs), a novel framework for learning concept-grounded
representations via weak supervision and implicit prototype representations.
iCSNs learn to bind conceptual information to specific prototype slots by
swapping the latent representations of paired images. This semantically
grounded and discrete latent space facilitates human understanding and
human-machine interaction. We support this claim by conducting experiments on
our novel data set "Elementary Concept Reasoning" (ECR), focusing on visual
concepts shared by geometric objects.
