Interlingual Homonymy Hinders Communication when a Person Reads Foreign Words from the Lips (from the Position of a Native Russian Speaker)
Abstract
Of all the sounds perceived by a person, the most important for him or her, of course, are the sounds of speech; these sounds allow people to communicate with each other. A person may become socially isolated in case he fails to communicate with native or foreign speakers if he looses hearing. A prerequisite to integrate a hearing-impaired person into the society is to improve their ability to visual perceive speech, i.e. to read words from the speaker’s lips. A person can visually perceive articulated speech to communicate with other people; this is an effective but difficult way to communicate. This happens because the communication process is accompanied by a number of problems caused by homonymous words in speech. The formal similarity between the phonetic and articulation shells of words with different meanings is misleading. The similarity also suggests the words are interchangeable, so the meaning of the words gets distorted. The paper investigates lexical homonymy as a barrier between people to communicate when they read from lips. To investigate the intralingual and interlingual lexical homonyms, we compare speech elements of the Russian language and a number of foreign languages. We review the dictionaries of the most common languages in order to identify interlingual homonyms in the dictionaries, similar to the words of the Russian language with phonetic and articulatory shells. We compared homonyms of these languages to analyze the issues of formal similarity and differences at the lexical level of homonyms. We present an example of how to compute the probability to correctly recognize homonymous words of the Russian language by their graphemes, phonemes, and visemes. We draw attention to read intra-language homonyms from the lips. For this we worked out an example of the homonyms of Russian language. Also, we emphasize inter-language homonyms as people verbally communicate in different language systems. We briefly describe a multimedia program to teach users to read from lips. The program reveals the nuances of words with similar visemes of different language systems and controls if the users correctly recognize the visemes.
References
[2] Sadokhin A.P. Vvedeniye v teoriyu mezhkul'turnoy kommunikatsii [Introduction to the Theory of Intercultural Communication. 2nd ed. Moscow: KnoRus; 2017. Available at: https://elibrary.ru/item.asp?id=27643903 (accessed 20.07.2020). (In Russ.)
[3] Gomon D.N. Omonimiya kak simmetriya morfem [Homonymy as Symmetry of Morphemes]. In: G. I. Shevchenko, G. I. Tananushki (ed.) Studia philological. 2015; (8):19-25. Minsk. Available at: http://graecolatini.bsu.by/htm-articles/studia-8-2015/gomon-2015.pdf (accessed 20.07.2020). (In Russ.)
[4] Vlček J. Úskalí ruské slovní zásoby. Slovník rusko-české homonymie a paronymie. Praha: Svět sovětů 1966. (In Czech)
[5] Stanishevska O. Źródła polsko-ukraińskich homonimów i paronimów. In: Proceedings of the VI International Sci-entific and Practical Conference “Current Trends in Young Scientists'Research” (April 11, 2019). Zhytomyr: ZSTU; 2019. p. 326-332. Available at: https://conf.ztu.edu.ua/wp-content/uploads/2019/06/309.pdf (accessed 20.07.2020). (In Polish)
[6] Khutsishvili S.D. Slavyanskiye mezh"yazykovyye omonimy: diss. ... d-ra filol. nauk [Slavic interlanguage homonyms: diss. ... Dr.Sci. (Philology)]. Tbilisi; 2010. Available at: https://newtsutest.tsu.ge/assets/media/files/48/disertaciebi/Sofo_Khutsishvili.pdf (accessed 20.07.2020). (In Russ.)
[7] Kusal'K.Ch. Russko-pol’skaja mezhjazykovaja omonimija i paronimija: diss. ... d-ra filol. nauk [Russian-Polish interlingual homonymy and paronymy: diss. ... Dr.Sci. (Philology)]. Saint Petersburg; 2005. (In Russ.)
[8] Fedorchuk E.V. Mezhjazykovaja omonimija i paronimija v blizkorodstvennyh jazykah: na materiale russkogo i ukrainskogo jazykov: diss. ... kand. fil. nauk [Interlanguage homonymy and paronymy in closely related languages: on the material of Russian and Ukrainian languages: diss. ... Ph.D. (Philology)]. Moscow; 2001. (In Russ.)
[9] Grigorjanova T.V. K voprosu o mezh"yazykovykh omonimakh v slavyanskikh yazykakh (na materiale slovatskogo, cheshskogo i russkogo yazykov) [On the Problem of Interlingual Homonyms in Slavic Languages (Using the Material of Slovak, Czech and Russian Languages)]. Filologicheskii klass = Philological Class. 2015; (4):23-26. Available at: https://elibrary.ru/item.asp?id=32296716 (accessed 20.07.2020). (In Russ.)
[10] Vasilyeva E.A., Konecny J. Cheshsko-Russkaja omonimija kak lingvodidakticheskaja problema [Czech-Russian Homonymy as a Didactic Problem]. XLinguae Journal. 2015; 8(1):57-67. (In Russ., abstract in Eng.) DOI: https://doi.org/10.17846/XL.2015.08.01.57-67
[11] Kaleta R. Białorusko-polska homonimia międzyjęzykowa: praca doktorska. Warszawa; 2011. Available at: https://depotuw.ceon.pl/bitstream/handle/item/314/Rados%c5%82aw_Kaleta_DR.pdf?sequence=1 (accessed 20.07.2020). (In Polish)
[12] Bel'tyukov V.I. Chteniye s gub [Reading from the lips]. Moscow: Pedagogika; 1970. (In Russ.)
[13] Mett A.I., Nikitina N.A. Zritel'noye vospriyatiye ust-noy rechi [Visual Perception of the Oral Speech]. Moscow: Prosveshcheniye; 1965. (In Russ.)
[14] Yuzhannikova M.A. Fenomen dvusmyslennosti kak osnovaniye stilisticheskikh priyemov v sov-remennom russkom yazyke: diss. … kand. filol. nauk [The phenomenon of ambiguity as the basis of stylistic devices in modern Russian: diss. ... Ph.D. (Philology)]. Krasnoyarsk; 2015. (In Russ.)
[15] Murashov A., Shmatko N. Ambiguity: Speech Mistake and "Language Game". Ural Philological Herald. Series Language. System. Personality: The Linguistics of Creativity. 2016; (2):169-175. Available at: https://elibrary.ru/item.asp?id=26465433 (accessed 20.07.2020). (In Russ., abstract in Eng.)
[16] Sereda P.V. Omonimiya i mnogoznachimost'kak instrument yazykovoy igry: na materiale russkogo i angliyskogo yazykov: diss. … kand. filol. nauk [Homonymy and polysemy as an instrument of a language game: on the material of the Russian and English languages: diss. ... Ph.D. (Philology)]. Krasnodar; 2013. (In Russ.)
[17] Myasoedova M.A., Myasoedova Z.P. Analysis of Visual Recognition by Man Isolated Words of Oral Russian Speech in the Conditions of Uncertainty. Sovremennye informacionnye tehnologii i IT-obrazovanie = Modern Information Technologies and IT-Education. 2017; 13(1):199-205. (In Russ., abstract in Eng.) DOI: https://doi.org/10.25559/SITITO.2017.1.413
[18] Myasoedova M.A., Myasoedova Z.P., Farkhadov M.P. Articulatory uncertainty as a result of visual recognition of modified sounds in Russian speech. In: 2017 IEEE 11th International Conference on Application of Information and Communication Technologies (AICT). Moscow, Russia; 2017. p. 1-4. (In Eng.) DOI: https://doi.org/10.1109/ICAICT.2017.8686938
[19] Auer E.T. Investigating speechreading and deafness. Journal of the American Academy of Audiology. 2010; 21(3):163-168. (In Eng.) DOI: https://doi.org/10.3766/jaaa.21.3.4
[20] Liew A.W.-C., Wang S. Visual Speech Recognition: Lip Segmentation and Mapping. New York: IGI Global; 2009. (In Eng.) DOI: https://doi.org/10.4018/978-1-60566-186-5
[21] Afouras T., Chung J.S., Senior A., Vinyals O., Zisserman A. Deep Audio-visual Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018; 1-11. (In Eng.) DOI: https://doi.org/10.1109/TPAMI.2018.2889052
[22] Chung J.S., Senior A., Vinyals O., Zisserman A. Lip Reading Sentences in the Wild. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI; 2017. p. 3444-3453. (In Eng.) DOI: https://doi.org/10.1109/CVPR.2017.367
[23] Almajai I., Cox S., Harvey R., Lan Y. Improved speaker independent lip reading using speaker adaptive training and deep neural networks. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shanghai; 2016. p. 2722-2726. (In Eng.) DOI: https://doi.org/10.1109/ICASSP.2016.7472172
[24] Bear H.L., Harvey R. Decoding visemes: Improving machine lip-reading. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shanghai; 2016. p. 2009-2013. (In Eng.) DOI: https://doi.org/10.1109/ICASSP.2016.7472029
[25] Tamura S., Ninomiya H., Kitaoka N., Osuga S., Iribe Y., Takeda K., Hayamizu S. Audio-visual speech recognition using deep bottleneck features and high-performance lipreading. In: 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). Hong Kong; 2015. p. 575-582. (In Eng.) DOI: https://doi.org/10.1109/APSIPA.2015.7415335
[26] Sui C., Bennamoun M., Togneri R. Listening with Your Eyes: Towards a Practical Visual Speech Recognition System Using Deep Boltzmann Machines. In: 2015 IEEE International Conference on Computer Vision (ICCV). Santiago; 2015. p. 154-162. (In Eng.) DOI: https://doi.org/10.1109/ICCV.2015.26
[27] Ding R., Pang C., Liu H. Audio-Visual Keyword Spotting Based on Multidimensional Convolutional Neural Network. In: 2018 25th IEEE International Conference on Image Processing (ICIP). Athens; 2018. p. 4138-4142. (In Eng.) DOI: https://doi.org/10.1109/ICIP.2018.8451096
[28] Chung J.S., Zisserman A. Learning to lip read words by watching videos. Computer Vision and Image Understanding. 2018; 173:76-85. (In Eng.) DOI: https://doi.org/10.1016/j.cviu.2018.02.001
[29] Myasoyedova M.A., Myasoedova Z.P. Computer assessment of how well a person visually recognizes verbal Russian speech. In: 2018 IEEE 12th International Conference on Application of Information and Communication Technologies (AICT). Almaty, Kazakhstan; 2018. p. 1-5. (In Eng.) DOI: https://doi.org/10.1109/ICAICT.2018.8747072
[30] Myasoyedova M.A., Myasoedova Z.P., Petukhova N.V. Vizual'noye vospriyatiye ustnoy rechi i obuchayushchaya programma "Chitayem s gub" [Visual Perception of Oral Speech and Software "Chitayem s gub"]. Moscow: Buki Vedi; 2015. (In Russ.)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Publication policy of the journal is based on traditional ethical principles of the Russian scientific periodicals and is built in terms of ethical norms of editors and publishers work stated in Code of Conduct and Best Practice Guidelines for Journal Editors and Code of Conduct for Journal Publishers, developed by the Committee on Publication Ethics (COPE). In the course of publishing editorial board of the journal is led by international rules for copyright protection, statutory regulations of the Russian Federation as well as international standards of publishing.
Authors publishing articles in this journal agree to the following: They retain copyright and grant the journal right of first publication of the work, which is automatically licensed under the Creative Commons Attribution License (CC BY license). Users can use, reuse and build upon the material published in this journal provided that such uses are fully attributed.
