This chapter presents two examples of human action modeling, namely the modeling of human facial expression intensity and body limb actions. It describes a tracking system that is capable of locating the head and hand positions of moving humans have been developed. The chapter explores the motion trajectories of humans in the scene by using support vector classification and explains principal component analysis (PCA) and independent component analysis for data reduction. PCA is used to map the net motion into a low dimension emotion space, and the degree of expression is estimated in that space. The expression intensity of a frame in a sequence is estimated by using the correlation property of PCA. In-plane head motion and small out-of-plane motion are allowed, and image normalization is performed in order to separate the non-rigid facial expression from rigid head motion. The intensity of the facial expression was then extracted from the trajectory of the feature points using isometric feature mapping.