  The objective of offline RL is to learn optimal policies when a fixed
exploratory demonstrations data-set is available and sampling additional
observations is impossible (typically if this operation is either costly or
rises ethical questions). In order to solve this problem, off the shelf
approaches require a properly defined cost function (or its evaluation on the
provided data-set), which are seldom available in practice. To circumvent this
issue, a reasonable alternative is to query an expert for few optimal
demonstrations in addition to the exploratory data-set. The objective is then
to learn an optimal policy w.r.t. the expert's latent cost function. Current
solutions either solve a behaviour cloning problem (which does not leverage the
exploratory data) or a reinforced imitation learning problem (using a fixed
cost function that discriminates available exploratory trajectories from expert
ones). Inspired by the success of IRL techniques in achieving state of the art
imitation performances in online settings, we exploit GAN based data
augmentation procedures to construct the first offline IRL algorithm. The
obtained policies outperformed the aforementioned solutions on multiple OpenAI
gym environments.
