  Human-robot teaming is one of the most important applications of artificial
intelligence in the fast-growing field of robotics. For effective teaming, a
robot must not only maintain a behavioral model of its human teammates to
project the team status, but also be aware that its human teammates'
expectation of itself. Being aware of the human teammates' expectation leads to
robot behaviors that better align with human expectation, thus facilitating
more efficient and potentially safer teams. Our work addresses the problem of
human-robot cooperation with the consideration of such teammate models in
sequential domains by leveraging the concept of plan explicability. In plan
explicability, however, the human is considered solely as an observer. In this
paper, we extend plan explicability to consider interactive settings where
human and robot behaviors can influence each other. We term this new measure as
Interactive Plan Explicability. We compare the joint plan generated with the
consideration of this measure using the fast forward planner (FF) with the plan
created by FF without such consideration, as well as the plan created with
actual human subjects. Results indicate that the explicability score of plans
generated by our algorithm is comparable to the human plan, and better than the
plan created by FF without considering the measure, implying that the plans
created by our algorithms align better with expected joint plans of the human
during execution. This can lead to more efficient collaboration in practice.
