  We give a model-based agent that builds a Python program representing its
knowledge of the world based on its interactions with the environment. The
world model tries to explain its interactions, while also being optimistic
about what reward it can achieve. We define this optimism as a logical
constraint between a program and a planner. We study our agent on gridworlds,
and on task planning, finding our approach is more sample-efficient compared to
deep RL, more compute-efficient compared to ReAct-style agents, and that it can
transfer its knowledge across environments by editing its code.
