Towards contextbased visual feedback Recognition for Embodied Agents.pdf

MITSUBISHI ELECTRIC RESEARCH LABORATORIES Towards Context-Based Visual Feedback Recognition for Embodied Agents Louis-Philippe Morency, Candace L. Sidner, Trevor Darrell TR2004-145 April 2005 Abstract Head pose and gesture offer several key conversational grounding cues and are used extensively in face-to-face interaction among people. We investigate how contextural information can im- prove visual recognition of feedback gestures during interactions with embodied conversational agents. We present a visual recognition model that integrates cues from the spoken dialogue of an embodied agent with direct observation of a users? head pose. In preliminary experiments us- ing a discriminative framework, contextual information improved the performance of head nod detection. AISB Symposium on Conversational Informatics, April 2005 This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c?Mitsubishi Electric Research Laboratories, Inc., 2005 201 Broadway, Cambridge, Massachusetts 02139 MERLCoverPageSide2 Towards Context-Based Visual Feedback Recognition for Embodied Agents Louis-Philippe Morency ? Candace Sidner ? Trevor Darrell ? ? Computer Sciences and Artificial Intelligence Laboratory Massachussetts Institute of Technology Cambridge, MA 02139, USA {lmorency,trevor}@ ? Mitsubishi Electric Research Laboratories (MERL) Cambridge, MA 02139, USA sidner