  This paper studies semantic-aware communication for remote estimation of
multiple Markov sources over a lossy and rate-constrained channel. Unlike most
existing studies that treat all source states equally, we exploit the semantics
of information and consider that the remote actuator has different tolerances
for the estimation errors of different states. We aim to find an optimal
scheduling policy that minimizes the long-term state-dependent costs of
estimation errors under a transmission frequency constraint. We theoretically
show the structure of the optimal policy by leveraging the average-cost
Constrained Markov Decision Process (CMDP) theory and the Lagrangian dynamic
programming. By exploiting the optimal structural results, we develop a novel
policy search algorithm, termed intersection search plus relative value
iteration (Insec-RVI), that can find the optimal policy using only a few
iterations. To avoid the ``curse of dimensionality'' of MDPs, we propose an
online low-complexity drift-plus-penalty (DPP) scheduling algorithm based on
the Lyapunov optimization theorem. We also design an efficient average-cost
Q-learning algorithm to estimate the optimal policy without knowing a priori
the channel and source statistics. Numerical results show that continuous
transmission is inefficient, and remarkably, our semantic-aware policies can
attain the optimum by strategically utilizing fewer transmissions by exploiting
the timing of the important information.
