  Recent years have seen human robot collaboration (HRC) quickly emerged as a
hot research area at the intersection of control, robotics, and psychology.
While most of the existing work in HRC focused on either low-level human-aware
motion planning or HRC interface design, we are particularly interested in a
formal design of HRC with respect to high-level complex missions, where it is
of critical importance to obtain an accurate and meanwhile tractable human
model. Instead of assuming the human model is given, we ask whether it is
reasonable to learn human models from observed perception data, such as the
gesture, eye movements, head motions of the human in concern. As our initial
step, we adopt a partially observable Markov decision process (POMDP) model in
this work as mounting evidences have suggested Markovian properties of human
behaviors from psychology studies. In addition, POMDP provides a general
modeling framework for sequential decision making where states are hidden and
actions have stochastic outcomes. Distinct from the majority of POMDP model
learning literature, we do not assume that the state, the transition structure
or the bound of the number of states in POMDP model is given. Instead, we use a
Bayesian non-parametric learning approach to decide the potential human states
from data. Then we adopt an approach inspired by probably approximately correct
(PAC) learning to obtain not only an estimation of the transition probability
but also a confidence interval associated to the estimation. Then, the
performance of applying the control policy derived from the estimated model is
guaranteed to be sufficiently close to the true model. Finally, data collected
from a driver-assistance test-bed are used to train the model, which
illustrates the effectiveness of the proposed learning method.
