  Recent advances in reinforcement learning (RL) have increased the promise of
introducing cognitive assistance and automation to robot-assisted laparoscopic
surgery (RALS). However, progress in algorithms and methods depends on the
availability of standardized learning environments that represent skills
relevant to RALS. We present LapGym, a framework for building RL environments
for RALS that models the challenges posed by surgical tasks, and sofa_env, a
diverse suite of 12 environments. Motivated by surgical training, these
environments are organized into 4 tracks: Spatial Reasoning, Deformable Object
Manipulation & Grasping, Dissection, and Thread Manipulation. Each environment
is highly parametrizable for increasing difficulty, resulting in a high
performance ceiling for new algorithms. We use Proximal Policy Optimization
(PPO) to establish a baseline for model-free RL algorithms, investigating the
effect of several environment parameters on task difficulty. Finally, we show
that many environments and parameter configurations reflect well-known, open
problems in RL research, allowing researchers to continue exploring these
fundamental problems in a surgical context. We aim to provide a challenging,
standard environment suite for further development of RL for RALS, ultimately
helping to realize the full potential of cognitive surgical robotics. LapGym is
publicly accessible through GitHub (https://github.com/ScheiklP/lap_gym).
