  Runtime assurance (RTA) addresses the problem of keeping an autonomous system
safe while using an untrusted (or experimental) controller. This can be done
via logic that explicitly switches between the untrusted controller and a
safety controller, or logic that filters the input provided by the untrusted
controller. While several tools implement specific instances of RTAs, there is
currently no framework for evaluating different approaches. Given the
importance of the RTA problem in building safe autonomous systems, an
evaluation tool is needed. In this paper, we present the RTAEval framework as a
low code framework that can be used to quickly evaluate different RTA logics
for different types of agents in a variety of scenarios. RTAEval is designed to
quickly create scenarios, run different RTA logics, and collect data that can
be used to evaluate and visualize performance. In this paper, we describe
different components of RTAEval and show how it can be used to create and
evaluate scenarios involving multiple aircraft models.
