  Q-learning is widely recognized as an effective approach for synthesizing
controllers to achieve specific goals. However, handling challenges posed by
continuous state-action spaces remains an ongoing research focus. This paper
presents a systematic analysis that highlights a major drawback in space
discretization methods. To address this challenge, the paper proposes a
symbolic model that represents behavioral relations, such as alternating
simulation from abstraction to the controlled system. This relation allows for
seamless application of the synthesized controller based on abstraction to the
original system. Introducing a novel Q-learning technique for symbolic models,
the algorithm yields two Q-tables encoding optimal policies. Theoretical
analysis demonstrates that these Q-tables serve as both upper and lower bounds
on the Q-values of the original system with continuous spaces. Additionally,
the paper explores the correlation between the parameters of the space
abstraction and the loss in Q-values. The resulting algorithm facilitates
achieving optimality within an arbitrary accuracy, providing control over the
trade-off between accuracy and computational complexity. The obtained results
provide valuable insights for selecting appropriate learning parameters and
refining the controller. The engineering relevance of the proposed Q-learning
based symbolic model is illustrated through two case studies.
