  This paper focuses on finding reinforcement learning policies for control
systems with hard state and action constraints. Despite its success in many
domains, reinforcement learning is challenging to apply to problems with hard
constraints, especially if both the state variables and actions are
constrained. Previous works seeking to ensure constraint satisfaction, or
safety, have focused on adding a projection step to a learned policy. Yet, this
approach requires solving an optimization problem at every policy execution
step, which can lead to significant computational costs.
  To tackle this problem, this paper proposes a new approach, termed Vertex
Networks (VNs), with guarantees on safety during exploration and on learned
control policies by incorporating the safety constraints into the policy
network architecture. Leveraging the geometric property that all points within
a convex set can be represented as the convex combination of its vertices, the
proposed algorithm first learns the convex combination weights and then uses
these weights along with the pre-calculated vertices to output an action. The
output action is guaranteed to be safe by construction. Numerical examples
illustrate that the proposed VN algorithm outperforms vanilla reinforcement
learning in a variety of benchmark control tasks.
