  De gustibus non est disputandum ("there is no accounting for others' tastes")
is a common Latin maxim describing how many solutions in life are determined by
people's personal preferences. Many household tasks, in particular, can only be
considered fully successful when they account for personal preferences such as
the visual aesthetic of the scene. For example, setting a table could be
optimized by arranging utensils according to traditional rules of Western table
setting decorum, without considering the color, shape, or material of each
object, but this may not be a completely satisfying solution for a given
person. Toward this end, we present DegustaBot, an algorithm for visual
preference learning that solves household multi-object rearrangement tasks
according to personal preference. To do this, we use internet-scale pre-trained
vision-and-language foundation models (VLMs) with novel zero-shot visual
prompting techniques. To evaluate our method, we collect a large dataset of
naturalistic personal preferences in a simulated table-setting task, and
conduct a user study in order to develop two novel metrics for determining
success based on personal preference. This is a challenging problem and we find
that 50% of our model's predictions are likely to be found acceptable by at
least 20% of people.
