  In real-world decision optimization, often multiple competing objectives must
be taken into account. Following classical reinforcement learning, these
objectives have to be combined into a single reward function. In contrast,
multi-objective reinforcement learning (MORL) methods learn from vectors of
per-objective rewards instead. In the case of multi-policy MORL, sets of
decision policies for various preferences regarding the conflicting objectives
are optimized. This is especially important when target preferences are not
known during training or when preferences change dynamically during
application. While it is, in general, straightforward to extend a
single-objective reinforcement learning method for MORL based on linear
scalarization, solutions that are reachable by these methods are limited to
convex regions of the Pareto front. Non-linear MORL methods like Thresholded
Lexicographic Ordering (TLO) are designed to overcome this limitation.
Generalized MORL methods utilize function approximation to generalize across
objective preferences and thereby implicitly learn multiple policies in a
data-efficient manner, even for complex decision problems with high-dimensional
or continuous state spaces. In this work, we propose \textit{generalized
Thresholded Lexicographic Ordering} (gTLO), a novel method that aims to combine
non-linear MORL with the advantages of generalized MORL. We introduce a deep
reinforcement learning realization of the algorithm and present promising
results on a standard benchmark for non-linear MORL and a real-world
application from the domain of manufacturing process control.
