  Signal temporal logic (STL) is an expressive language to specify time-bound
real-world robotic tasks and safety specifications. Recently, there has been an
interest in learning optimal policies to satisfy STL specifications via
reinforcement learning (RL). Learning to satisfy STL specifications often needs
a sufficient length of state history to compute reward and the next action. The
need for history results in exponential state-space growth for the learning
problem. Thus the learning problem becomes computationally intractable for most
real-world applications. In this paper, we propose a compact means to capture
state history in a new augmented state-space representation. An approximation
to the objective (maximizing probability of satisfaction) is proposed and
solved for in the new augmented state-space. We show the performance bound of
the approximate solution and compare it with the solution of an existing
technique via simulations.
