  Handling trust is one of the core requirements for facilitating effective
interaction between the human and the AI agent. Thus, any decision-making
framework designed to work with humans must possess the ability to estimate and
leverage human trust. In this paper, we propose a mental model based theory of
trust that not only can be used to infer trust, thus providing an alternative
to psychological or behavioral trust inference methods, but also can be used as
a foundation for any trust-aware decision-making frameworks. First, we
introduce what trust means according to our theory and then use the theory to
define trust evolution, human reliance and decision making, and a formalization
of the appropriate level of trust in the agent. Using human subject studies, we
compare our theory against one of the most common trust scales (Muir scale) to
evaluate 1) whether the observations from the human studies match our proposed
theory and 2) what aspects of trust are more aligned with our proposed theory.
