Toward Natural Gesture/Speech Control of a Large Display

Details Toward Natural Gesture/Speech Control of a Large Display Toward Natural Gesture/Speech Control of a Large Display

2001-05-17

arxiv.org/abs/cs/0105026v1

Computer Science

Computer Vision and Pattern Recognition

Engineering for Human-Computer Interaction (EHCI'01),Toronto, Canada. May 11-14, 2001. Lecture Notes in Computer Science, Spri

Scientific paper

In recent years because of the advances in computer vision research, free hand gestures have been explored as means of human-computer interaction (HCI). Together with improved speech processing technology it is an important step toward natural multimodal HCI. However, inclusion of non-predefined continuous gestures into a multimodal framework is a challenging problem. In this paper, we propose a structured approach for studying patterns of multimodal language in the context of a 2D-display control. We consider systematic analysis of gestures from observable kinematical primitives to their semantics as pertinent to a linguistic structure. Proposed semantic classification of co-verbal gestures distinguishes six categories based on their spatio-temporal deixis. We discuss evolution of a computational framework for gesture and speech integration which was used to develop an interactive testbed (iMAP). The testbed enabled elicitation of adequate, non-sequential, multimodal patterns in a narrative mode of HCI. Conducted user studies illustrate significance of accounting for the temporal alignment of gesture and speech parts in semantic mapping. Furthermore, co-occurrence analysis of gesture/speech production suggests syntactic organization of gestures at the lexical level.

Affiliated with

Also associated with

No associations

Rating

Toward Natural Gesture/Speech Control of a Large Display does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Toward Natural Gesture/Speech Control of a Large Display, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Toward Natural Gesture/Speech Control of a Large Display will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-184619