  We propose a safe exploration algorithm for deterministic Markov Decision
Processes with unknown transition models. Our algorithm guarantees safety by
leveraging Lipschitz-continuity to ensure that no unsafe states are visited
during exploration. Unlike many other existing techniques, the provided safety
guarantee is deterministic. Our algorithm is optimized to reduce the number of
actions needed for exploring the safe space. We demonstrate the performance of
our algorithm in comparison with baseline methods in simulation on navigation
tasks.
