  Tools have become a mainstay of LLMs, allowing them to retrieve knowledge not
in their weights, to perform tasks on the web, and even to control robots.
However, most ontologies and surveys of tool-use have assumed the core
challenge for LLMs is choosing the tool. Instead, we introduce a framework for
tools more broadly which guides us to explore a model's ability to detect
"silent" tool errors, and reflect on how to plan. This more directly aligns
with the increasingly popular use of models as tools. We provide an initial
approach to failure recovery with promising results both on a controlled
calculator setting and embodied agent planning.
