  Reinforcement learning (RL) has been extensively researched for enhancing
human-environment interactions in various human-centric tasks, including
e-learning and healthcare. Since deploying and evaluating policies online are
high-stakes in such tasks, off-policy evaluation (OPE) is crucial for inducing
effective policies. In human-centric environments, however, OPE is challenging
because the underlying state is often unobservable, while only aggregate
rewards can be observed (students' test scores or whether a patient is released
from the hospital eventually). In this work, we propose a human-centric OPE
(HOPE) to handle partial observability and aggregated rewards in such
environments. Specifically, we reconstruct immediate rewards from the
aggregated rewards considering partial observability to estimate expected total
returns. We provide a theoretical bound for the proposed method, and we have
conducted extensive experiments in real-world human-centric tasks, including
sepsis treatments and an intelligent tutoring system. Our approach reliably
predicts the returns of different policies and outperforms state-of-the-art
benchmarks using both standard validation methods and human-centric
significance tests.
