  Most Reinforcement Learning (RL) environments are created by adapting
existing physics simulators or video games. However, they usually lack the
flexibility required for analyzing specific characteristics of RL methods often
relevant to research. This paper presents Craftium, a novel framework for
exploring and creating rich 3D visual RL environments that builds upon the
Minetest game engine and the popular Gymnasium API. Minetest is built to be
extended and can be used to easily create voxel-based 3D environments (often
similar to Minecraft), while Gymnasium offers a simple and common interface for
RL research. Craftium provides a platform that allows practitioners to create
fully customized environments to suit their specific research requirements,
ranging from simple visual tasks to infinite and procedurally generated worlds.
We also provide five ready-to-use environments for benchmarking and as examples
of how to develop new ones. The code and documentation are available at
https://github.com/mikelma/craftium/.
