Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures

Details Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures

2002-11-05

arxiv.org/abs/cs/0211005v1

S. Kettebekov, M. Yeasin, and R. Sharma, "Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures," presente

Computer Science

Computer Vision and Pattern Recognition

Alternative see: http://vision.cse.psu.edu/kettebek/academ/publications.htm

Scientific paper

Although speech and gesture recognition has been studied extensively, all the successful attempts of combining them in the unified framework were semantically motivated, e.g., keyword-gesture cooccurrence. Such formulations inherited the complexity of natural language processing. This paper presents a Bayesian formulation that uses a phenomenon of gesture and speech articulation for improving accuracy of automatic recognition of continuous coverbal gestures. The prosodic features from the speech signal were coanalyzed with the visual signal to learn the prior probability of co-occurrence of the prominent spoken segments with the particular kinematical phases of gestures. It was found that the above co-analysis helps in detecting and disambiguating visually small gestures, which subsequently improves the rate of continuous gesture recognition. The efficacy of the proposed approach was demonstrated on a large database collected from the weather channel broadcast. This formulation opens new avenues for bottom-up frameworks of multimodal integration.

Affiliated with

Also associated with

No associations

Rating

Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-635931