  A core problem in cognitive science and machine learning is to understand how
humans derive semantic representations from perceptual objects, such as color
from an apple, pleasantness from a musical chord, or seriousness from a face.
Markov Chain Monte Carlo with People (MCMCP) is a prominent method for studying
such representations, in which participants are presented with binary choice
trials constructed such that the decisions follow a Markov Chain Monte Carlo
acceptance rule. However, while MCMCP has strong asymptotic properties, its
binary choice paradigm generates relatively little information per trial, and
its local proposal function makes it slow to explore the parameter space and
find the modes of the distribution. Here we therefore generalize MCMCP to a
continuous-sampling paradigm, where in each iteration the participant uses a
slider to continuously manipulate a single stimulus dimension to optimize a
given criterion such as 'pleasantness'. We formulate both methods from a
utility-theory perspective, and show that the new method can be interpreted as
'Gibbs Sampling with People' (GSP). Further, we introduce an aggregation
parameter to the transition step, and show that this parameter can be
manipulated to flexibly shift between Gibbs sampling and deterministic
optimization. In an initial study, we show GSP clearly outperforming MCMCP; we
then show that GSP provides novel and interpretable results in three other
domains, namely musical chords, vocal emotions, and faces. We validate these
results through large-scale perceptual rating experiments. The final
experiments use GSP to navigate the latent space of a state-of-the-art image
synthesis network (StyleGAN), a promising approach for applying GSP to
high-dimensional perceptual spaces. We conclude by discussing future cognitive
applications and ethical implications.
