  In this study, we explore the sophisticated domain of task planning for
robust household embodied agents, with a particular emphasis on the intricate
task of selecting substitute objects. We introduce the CommonSense Object
Affordance Task (COAT), a novel framework designed to analyze reasoning
capabilities in commonsense scenarios. This approach is centered on
understanding how these agents can effectively identify and utilize alternative
objects when executing household tasks, thereby offering insights into the
complexities of practical decision-making in real-world environments.Drawing
inspiration from human decision-making, we explore how large language models
tackle this challenge through three meticulously crafted commonsense
question-and-answer datasets, featuring refined rules and human annotations.
Our evaluation of state-of-the-art language models on these datasets sheds
light on three pivotal considerations: 1) aligning an object's inherent utility
with the task at hand, 2) navigating contextual dependencies (societal norms,
safety, appropriateness, and efficiency), and 3) accounting for the current
physical state of the object. To maintain accessibility, we introduce five
abstract variables reflecting an object's physical condition, modulated by
human insights to simulate diverse household scenarios. Our contributions
include insightful Object-Utility mappings addressing the first consideration
and two extensive QA datasets (15k and 130k questions) probing the intricacies
of contextual dependencies and object states. The datasets, along with our
findings, are accessible at: \url{https://github.com/com-phy-affordance/COAT}.
This research not only advances our understanding of physical commonsense
reasoning in language models but also paves the way for future improvements in
household agent intelligence.
