  As artificial intelligence research advances, the platforms used to evaluate
AI agents need to adapt and grow to continue to challenge them. We present the
Polycraft World AI Lab (PAL), a task simulator with an API based on the
Minecraft mod Polycraft World. Our platform is built to allow AI agents with
different architectures to easily interact with the Minecraft world, train and
be evaluated in multiple tasks. PAL enables the creation of tasks in a flexible
manner as well as having the capability to manipulate any aspect of the task
during an evaluation. All actions taken by AI agents and external actors
(non-player-characters, NPCs) in the open-world environment are logged to
streamline evaluation. Here we present two custom tasks on the PAL platform,
one focused on multi-step planning and one focused on navigation, and
evaluations of agents solving them. In summary, we report a versatile and
extensible AI evaluation platform with a low barrier to entry for AI
researchers to utilize.
