  We propose an algorithm for tabular episodic reinforcement learning with
constraints. We provide a modular analysis with strong theoretical guarantees
for settings with concave rewards and convex constraints, and for settings with
hard constraints (knapsacks). Most of the previous work in constrained
reinforcement learning is limited to linear constraints, and the remaining work
focuses on either the feasibility question or settings with a single episode.
Our experiments demonstrate that the proposed algorithm significantly
outperforms these approaches in existing constrained episodic environments.
