  Being able to detect and recognize human activities is essential for several
applications, including personal assistive robotics. In this paper, we perform
detection and recognition of unstructured human activity in unstructured
environments. We use a RGBD sensor (Microsoft Kinect) as the input sensor, and
compute a set of features based on human pose and motion, as well as based on
image and pointcloud information. Our algorithm is based on a hierarchical
maximum entropy Markov model (MEMM), which considers a person's activity as
composed of a set of sub-activities. We infer the two-layered graph structure
using a dynamic programming approach. We test our algorithm on detecting and
recognizing twelve different activities performed by four people in different
environments, such as a kitchen, a living room, an office, etc., and achieve
good performance even when the person was not seen before in the training set.
