  Real world applications of Reinforcement Learning (RL) are often partially
observable, thus requiring memory. Despite this, partial observability is still
largely ignored by contemporary RL benchmarks and libraries. We introduce
Partially Observable Process Gym (POPGym), a two-part library containing (1) a
diverse collection of 15 partially observable environments, each with multiple
difficulties and (2) implementations of 13 memory model baselines -- the most
in a single RL library. Existing partially observable benchmarks tend to fixate
on 3D visual navigation, which is computationally expensive and only one type
of POMDP. In contrast, POPGym environments are diverse, produce smaller
observations, use less memory, and often converge within two hours of training
on a consumer-grade GPU. We implement our high-level memory API and memory
baselines on top of the popular RLlib framework, providing plug-and-play
compatibility with various training algorithms, exploration strategies, and
distributed training paradigms. Using POPGym, we execute the largest comparison
across RL memory models to date. POPGym is available at
https://github.com/proroklab/popgym.
