一种应用虚拟发音头像的普通话聋儿言语康复系统.pdf

第 2 卷 第4期 集 成 技 术 Vol. 2 No. 4 2013年7月 JOURNAL OF INTEGRATION TECHNOLOGY Jul. 2013 一种应用虚拟发音头像的普通话聋儿言语康复系统 刘晓千  燕  楠  王  岚 （中国科学院深圳先进技术研究院 深圳 518055 ） 摘  要  本文介绍了一种新型的包含虚拟发音人头像的语言训练系统并对该系统在听障儿童语言康复训练中的实用性进 行了研究。该系统以虚拟三维发音人头像的形式再现了汉语中常见的20 个单音节字的发音动作，并将之用于语言康复 训练。系统运用EMA (三维电磁发音仪)设备采集内部及外部发音器官运动轨迹作为驱动数据，有效模拟了汉语发音。此 外，系统还包括一个听力训练模块和一个朗读训练模块，用于辅助聋儿的发音康复训练。文章对系统发音模拟功能的准 确性进行了人工评估，并应用系统对两名4 至 5 岁之间的聋儿进行了初步的康复训练。初步评估及训练的结果表明：系 统的发音模拟功能可有效模拟发音器官的动作，有助于提高听障儿童的语言能力；同时，系统的设计风格较易获得幼儿 认同。研究证明，本系统在听障儿童发音训练方面是有效的，具有较高的应用价值。 关键词  听障儿童；交互式言语康复系统；三维模拟发音指导；虚拟现实 An Interactive Speech Training System with Virtual Reality Articulation for Hearing Impaired Mandarin-speaking Children LIU Xiao-qian YAN Nan WANG Lan ( Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China ) Abstract In this study, a novel interactive speech training system based on virtual reality articulation was introduced and its efficiency in the rehabilitation of hearing impaired (HI) children was investigated as well. 20 meaningful words in Mandarin were presented by using a 3D talking head in this system for articulatory training. The Electro-Magnetic Articulography (EMA) and graphic transform technology were used to develop the reality inner articulatory movements of 3D talking head. Furthermore, relevant corpus was organized in a listening module and a speaking module of the system to help to recover the HI children’s language skills. Then, the accuracy of virtual reality (VR) articulatory movement model was evaluated through a series of experiments. Finally, a pilot test was performed o