  Safe exploration is a key to applying reinforcement learning (RL) in
safety-critical systems. Existing safe exploration methods guaranteed safety
under the assumption of regularity, and it has been difficult to apply them to
large-scale real problems. We propose a novel algorithm, SPO-LF, that optimizes
an agent's policy while learning the relation between a locally available
feature obtained by sensors and environmental reward/safety using generalized
linear function approximations. We provide theoretical guarantees on its safety
and optimality. We experimentally show that our algorithm is 1) more efficient
in terms of sample complexity and computational cost and 2) more applicable to
large-scale problems than previous safe RL methods with theoretical guarantees,
and 3) comparably sample-efficient and safer compared with existing advanced
deep RL methods with safety constraints.
