  Safety is essential for deploying Deep Reinforcement Learning (DRL)
algorithms in real-world scenarios. Recently, verification approaches have been
proposed to allow quantifying the number of violations of a DRL policy over
input-output relationships, called properties. However, such properties are
hard-coded and require task-level knowledge, making their application
intractable in challenging safety-critical tasks. To this end, we introduce the
Collection and Refinement of Online Properties (CROP) framework to design
properties at training time. CROP employs a cost signal to identify unsafe
interactions and use them to shape safety properties. Hence, we propose a
refinement strategy to combine properties that model similar unsafe
interactions. Our evaluation compares the benefits of computing the number of
violations using standard hard-coded properties and the ones generated with
CROP. We evaluate our approach in several robotic mapless navigation tasks and
demonstrate that the violation metric computed with CROP allows higher returns
and lower violations over previous Safe DRL approaches.
