SILVA, A. L. C. ; SA, T. M. ; DINIZ, R. S. ; FERREIRA, S. B. L. ; SIQUEIRA, S. W. M. ; BOURGUIGNON, SAULO CABRAL . Prescriptive and Semantic Analysis of an Automatic Sign Language Translation: Cases on VLibras Avatar Translation Using Video Interviews and Textual Interactions With a Chatbot. INTERACTING WITH COMPUTERS, v. 35, n. 2, march 2023, pp. 231-246. doi: 10.1093/iwc/iwac020
Prescriptive and Semantic Analysis of an Automatic Sign Language Translation: Cases on VLibras Avatar Translation Using Video Interviews and Textual Interactions With a Chatbot
Authors
André Luiz da Cunha Silva (UNIRIO)
Tatiane Militão de Sá (UFF)
Ruan Sousa Diniz (UFRJ)
Simone Bacellar Leal Ferreira (UNIRIO)
Sean Wolfgand Matsui Siqueira (UNIRIO)
Saulo Cabral Bourguignon (UFF)
Abstract
Algorithms designed to translate textual content into sign language (SL) expressed through avatars have been used to reduce accessibility barriers. Our research aimed to identify whether the VLibras tool, widely adopted on Brazilian government websites, is an effective accessibility solution for automatic translation into SL. It is an exploratory and applied qualitative research project involving a bibliographic review and support from expert interpreters. We conducted two experimental studies using sequential chronological cuts and applying prescriptive and semantic analyses. We present evidence that there is no actual translation into SL in the automatic translation process performed by the VLibras translation algorithm (TA) but only a transposition of part of the SL lexicon to the Portuguese morphosyntactic structure. The automatic translation of long texts and texts with complex syntactic structures results in excessive pauses and dactylology for words that have a sign registered in the basic SL dictionary. Using human–computer interaction concepts to evaluate automatic translation into sign language by the VLibras TA expands the existing theoretical discussion. It also contributes to minimizing communication problems caused by the discrepancy between the original message and the machine translation, a practical applicability of this study.
Highlights
- Using semiotic engineering property of communicability to evaluate automatic translation into sign language and the preservation of the original message at the end of the translation performed by the translation algorithm.
- Performing two complementary experimental studies using prescriptive and semantic analyses in sequential chronological cuts on videos and interactions with a chatbot.
- The research positively influences studies involving human–computer interaction and semiotic engineering theories, with practical applicability in improving machine translation tools.
Keywords:
Prescriptive and semantic analyses, Sign language, Automatic translation, Deaf, Accessibility,
Human–Computer Interaction
doi: 10.1093/iwc/iwac020
