  Join order selection (JOS) is the problem of ordering join operations to
minimize total query execution cost and it is the core NP-hard combinatorial
optimization problem of query optimization. In this paper, we present JoinGym,
a lightweight and easy-to-use query optimization environment for reinforcement
learning (RL) that captures both the left-deep and bushy variants of the JOS
problem. Compared to existing query optimization environments, the key
advantages of JoinGym are usability and significantly higher throughput which
we accomplish by simulating query executions entirely offline. Under the hood,
JoinGym simulates a query plan's cost by looking up intermediate result
cardinalities from a pre-computed dataset. We release a novel cardinality
dataset for $3300$ SQL queries based on real IMDb workloads which may be of
independent interest, e.g., for cardinality estimation. Finally, we extensively
benchmark four RL algorithms and find that their cost distributions are
heavy-tailed, which motivates future work in risk-sensitive RL. In sum, JoinGym
enables users to rapidly prototype RL algorithms on realistic database problems
without needing to setup and run live systems.
