The Role of Auditory and Visual Cue in Perception of Mandarin Tones by Children with Cochlear Implant

WANG Yi-nuo; MEI Man; LI Ji; TANG Ping; LI Shan-peng

doi:10.3969/j.issn.1672-4933.2023.02.015

您当前的位置：

首页 >

文章列表页 >

The Role of Auditory and Visual Cue in Perception of Mandarin Tones by Children with Cochlear Implant

Rehabilitation Education | 更新时间：2023-05-22

- The Role of Auditory and Visual Cue in Perception of Mandarin Tones by Children with Cochlear Implant
- Chinese Scientific Journal of Hearing and Speech Rehabilitation Vol. 21, Issue 2, Pages: 174-178(2023)
- 作者机构：
  
  南京理工大学外国语学院南京 210094
- 作者简介：
- 基金信息：
- DOI：10.3969/j.issn.1672-4933.2023.02.015
  CLC：
- Published：15 March 2023，
  
  Received：29 May 2022，
- 稿件说明：
移动端阅览
王一诺,梅满,李吉等.听觉和视觉线索在人工耳蜗植入儿童感知汉语声调中的作用[J].中国听力语言康复科学杂志,2023,21(02):174-178.

WANG Yi-nuo,MEI Man,LI Ji,et al.The Role of Auditory and Visual Cue in Perception of Mandarin Tones by Children with Cochlear Implant[J].Chinese Scientific Journal of Hearing and Speech Rehabilitation,2023,21(02):174-178.
王一诺,梅满,李吉等.听觉和视觉线索在人工耳蜗植入儿童感知汉语声调中的作用[J].中国听力语言康复科学杂志,2023,21(02):174-178. DOI： 10.3969/j.issn.1672-4933.2023.02.015.

WANG Yi-nuo,MEI Man,LI Ji,et al.The Role of Auditory and Visual Cue in Perception of Mandarin Tones by Children with Cochlear Implant[J].Chinese Scientific Journal of Hearing and Speech Rehabilitation,2023,21(02):174-178. DOI： 10.3969/j.issn.1672-4933.2023.02.015.

摘要

目的

考察在安静环境和噪声环境下，提供视觉线索是否能帮助人工耳蜗植入儿童准确辨识汉语声调。

方法

招募80名学龄前人工耳蜗植入儿童（研究组）和50名健听儿童（对照组），在安静及噪声环境下，通过听录音或看视频并选择与之对应的卡通图片方式，考察儿童在依靠听觉单一线索和依靠听、视觉双线索条件下声调的辨识情况。

结果

在安静及噪声环境下，视觉线索会帮助人工耳蜗植入儿童更好地辨认汉语声调，但效果取决于具体声调：在安静环境下，二声在音视频条件下识别率显著高于仅音频条件（

＜0.05）；在噪声环境下，一声和三声在音视频条件下识别率显著高于仅音频条件（

＜0.05）。

结论

人工耳蜗植入儿童可借助视觉线索（即说话人的面部表情）更准确地辨识汉语声调，为言语康复中声调的感知训练提供启示。

Abstract

Objective

This study investigated whether the additional visual cues increase the identifications of Chinese lexical tone by children with cochlear implants in quiet and noise environment.

Method

80 preschool children with cochlear implants and 50 normal hearing controls at the same age participated in the experiment of tone perception in which they were asked to choose the one of two cartoon pictures corresponding to the minimal tone pairs presented in auditory-only modality and auditory-visual modality. Comparison was made between auditory-only perception and auditory-visual perception of lexical tones in both quiet and noise environment.

Results

Visual modality did improve tone perception by children with cochlear implants in quiet and noise environment but on different tones; in quiet environment the accuracy of tone 2 was lower in auditory than auditory-visual modality significantly (

0.05) while the accuracy of T1 and T3 were significantly lower in auditory-only than auditory-visual modality in noise environment (

0.05).

Conclusion

The facilitation of facial expressions can be found in perception of certain tones by children with cochlear implant under quiet and noise environment

which can be used in speech rehabilitation of hearing-impaired children.

关键词

人工耳蜗植入儿童汉语声调听视觉感知噪声

Keywords

Children with cochlear implantsChinese lexical tonesAuditory-visual perceptionNoise

references

Gandour JT, Harshman RA. Crosslanguage Differences in Tone Perception: A Multidimensional Scaling Investigation[J]. Language & Speech,1978,21(1):1-33.

Vandali AE, Hoesel R. Enhancement of temporal cues to pitch in cochlear implants: Effects on pitch ranking[J]. The Journal of the Acoustical Society of America,2012,132(1):392-402.

Burnham D, Ciocca V, Stokes S. Auditory-visual perception of lexical tone[C]. European Conference on Eurospeech Scandinavia. DBLP,2001:395-398.

Mixdorff H, Yu H, Burnham D. Visual cues in Mandarin tone perception[C]. INTERSPEECH 2005 - Eurospeech, 9th European Conference on Speech Communication and Technology, Lisbon, Portugal, September 4-8, 2005. DBLP, 2005.

Massaro DW, Cohen MM, Tseng CY. The evaluation and integration of pitch height and pitch contour in lexical tone perception in Mandarin Chinese[J]. Journal of Chinese Linguistics,1985,13(2):267-289.

McDermott HJ, Looi V. Perception of complex signals, including musical sounds, with cochlear implants[C]. International congress series. Elsevier, 2004. 201-204.

Huang TS, Wang NM, Liu SY. Tone perception of Mandarin-speaking postlingually deaf implantees using the Nucleus 22-Channel Cochlear Mini System[J]. The Annals of otology, rhinology & laryngology. Supplement,1995,166:294-298.

崔丽丽.人工耳蜗植入儿童的声调识别及发声研究[D].华东师范大学,2011.

丁馨.人工耳蜗植入儿童声调及重音的感知与产出研究[D].南京师范大学,2018.

Garg S,Hamarneh G, Jongman A, et al. Computer-vision analysis reveals facial movements made during Mandarin tone production align with pitch trajectories[J].Speech Communication,2019,113:47-62.

Attina V, Gibert G, Vatikiotis-Bateson E, et al. Production of Mandarin lexical tones: Auditory and visual components[C]. Auditory-Visual Speech Processing 2010. AVSP, 2010:59-64.

Shaw JA, Chen WR, Proctor MI, et al. On the inter-dependence of tonal and vocalic production goals in Chinese[C]. 10th International Seminar on Speech Production. Cologne, Germany, 2014:395-398.

Chen TH, Massaro DW. Seeing pitch: Visual information for lexical tones of Mandarin-Chinese[J]. The Journal of the Acoustical Society of America,2008,123(4):2356-2366.

Han Y, Goudbeek M, Mos M, et al. Relative Contribution of Auditory and Visual Information to Mandarin Chinese Tone Identification by Native and Tone-naïve Listeners[J]. Language and speech,2020,63(4):856-876.

Smith D, Burnham D. Facilitation of Mandarin tone perception by visual speech in clear and degraded audio: implications for cochlear implants[J]. Journal of the Acoustical Society of America,2012,131(2):1480-1489.

胡涵,顾文涛,丁馨,等.人工耳蜗植入儿童在噪声环境与耳语条件下的汉语声调感知[J].南京师范大学文学院学报,2021(3):116-125.

Clumeck H. The acquisition of tone[M]. Child Phonology,1980.257-275.

Mao Y, Xu L. Lexical tone recognition in noise in normal-hearing children and prelingually deafened children with cochlear implants[J]. International Journal of Audiology, 2017, 56(sup2): S23-S30.

Schorr EA, Fox NA, Van WV, et al. Auditory-visual fusion in speech perception in children with cochlear implants[C]. The National Academy of Sciences of the United States of America, 2005,102(51): 18748-18750.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Tone Recognition between Preschool Children with Normal Hearing and Children with Cochlear Implant in Different Testing Conditions

The Paper Review of Hearing Loss Induced by Ototoxic Substances Combined with Noise

Related Author

MENG Chao

GUO Qian-qian

LV Jing

QI Bei-er

CHEN Xue-qing

QIU Xin

YANG Shi-ming

YU Ning

Related Institution

首都医科大学附属北京同仁医院/北京市耳鼻咽喉科研究所/耳鼻咽喉头颈外科教育部重点实验室（首都医科大学）

解放军总医院耳鼻咽喉头颈外科

⁰