  Although speech and gesture recognition has been studied extensively, all the
successful attempts of combining them in the unified framework were
semantically motivated, e.g., keyword-gesture cooccurrence. Such formulations
inherited the complexity of natural language processing. This paper presents a
Bayesian formulation that uses a phenomenon of gesture and speech articulation
for improving accuracy of automatic recognition of continuous coverbal
gestures. The prosodic features from the speech signal were coanalyzed with the
visual signal to learn the prior probability of co-occurrence of the prominent
spoken segments with the particular kinematical phases of gestures. It was
found that the above co-analysis helps in detecting and disambiguating visually
small gestures, which subsequently improves the rate of continuous gesture
recognition. The efficacy of the proposed approach was demonstrated on a large
database collected from the weather channel broadcast. This formulation opens
new avenues for bottom-up frameworks of multimodal integration.
