  The impressive recent performance of large language models has led many to
wonder to what extent they can serve as models of general intelligence or are
similar to human cognition. We address this issue by applying GPT-3.5 and GPT-4
to a classic problem in human inductive reasoning known as property induction.
Over two experiments, we elicit human judgments on a range of property
induction tasks spanning multiple domains. Although GPT-3.5 struggles to
capture many aspects of human behaviour, GPT-4 is much more successful: for the
most part, its performance qualitatively matches that of humans, and the only
notable exception is its failure to capture the phenomenon of premise
non-monotonicity. Our work demonstrates that property induction allows for
interesting comparisons between human and machine intelligence and provides two
large datasets that can serve as benchmarks for future work in this vein.
