  Human brains respond to semantic features of presented stimuli with different
neurons. It is then curious whether modern deep neural networks admit a similar
behavior pattern. Specifically, this paper finds a small cluster of neurons in
a diffusion model corresponding to a particular subject. We call those neurons
the concept neurons. They can be identified by statistics of network gradients
to a stimulation connected with the given subject. The concept neurons
demonstrate magnetic properties in interpreting and manipulating generation
results. Shutting them can directly yield the related subject contextualized in
different scenes. Concatenating multiple clusters of concept neurons can
vividly generate all related concepts in a single image. A few steps of further
fine-tuning can enhance the multi-concept capability, which may be the first to
manage to generate up to four different subjects in a single image. For
large-scale applications, the concept neurons are environmentally friendly as
we only need to store a sparse cluster of int index instead of dense float32
values of the parameters, which reduces storage consumption by 90\% compared
with previous subject-driven generation methods. Extensive qualitative and
quantitative studies on diverse scenarios show the superiority of our method in
interpreting and manipulating diffusion models.
