  Learning to use tools to solve a variety of tasks is an innate ability of
humans and has been observed of animals in the wild. However, the underlying
mechanisms that are required to learn to use tools are abstract and widely
contested in the literature. In this paper, we study tool use in the context of
reinforcement learning and propose a framework for analyzing generalization
inspired by a classic study of tool using behavior, the trap-tube task.
Recently, it has become common in reinforcement learning to measure
generalization performance on a single test set of environments. We instead
propose transfers that produce multiple test sets that are used to measure
specified types of generalization, inspired by abilities demonstrated by animal
and human tool users. The source code to reproduce our experiments is publicly
available at https://github.com/fomorians/gym_tool_use.
