Persisting Grammatical Errors of Machine Translation

Welnitzová, Katarína; Munková, Daša

doi:10.2478/jazcas-2025-0039

Abstract

The paper investigates the quality of machine translation (MT) and traces its development through two main approaches – Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) – by comparing English-to-Slovak outputs produced by Google Translate. The aim of the paper is to evaluate the quality of MT outputs from the point of view of two typologically different languages – English, a predominantly analytic language, and Slovak, a primarily synthetic language – using a sample of newspaper texts, which are often translated by machine due to their wide vocabulary and varied subject matter. The research results indicate that NMT (obtained in 2023), compared to its predecessor SMT (obtained in 2017), has significantly improved in almost all framework categories. The NMT output is much more fluent, sounding more natural and comprehensible. In contrast, shortcomings can be found in the category of syntactic-semantic correlativeness and lexical semantics. In such cases, neural MT may struggle to select the appropriate fit-in-context meaning; moreover, these lexemes can further shift the meaning of the entire sentence, clause, or even utterance.

References

ABSOLON, Jakub (2024): Education for the Needs of the Post-Localisation Era: Transforming the Translation Industry in the AI Age. In: Acta Anglica Tyrnaviensia, Vol. 2, pp. 9–14.
Search in Google Scholar Back to article
BEDNÁROVÁ-GIBOVÁ, Klaudia – GAVUROVÁ, Miroslava – GRESTY, Jonathan (2024): A Propaedeutics of Translation Studies. Prešov: Vydavateľstvo Prešovskej univerzity. 194 p.
Search in Google Scholar Back to article
BENKO, Ľubomír – MUNKOVÁ, Daša (2016): Application of POS Tagging in Machine Translation Evaluation. In: DIVAI 2016: 11th International Scientific Conference on Distance Learning in Applied Informatics. May 2–4, 2016, Štúrovo, Slovakia. Prague: Wolters Kluwer, pp. 471–489.
Search in Google Scholar Back to article
BENTIVOGLI, Luisa – BISAZZA, Arianna – CETTOLO, Mauro – FEDERICO, Marcello (2016): Neural versus phrase-based machine translation quality: a case study. In: J. Su – K. Duh – X. Carreras (eds.): Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin: Association for Computational Linguistics, pp. 257–267.
Search in Google Scholar Back to article
BESSENYEI, György (2017): Neural Machine Translation: The Rising Star. Memsource. Available at: www.memsource.com/blog/2024/09/19/neural-machine-translation-the-rising-star/?utm_source=mailchimp&utm_medium=email&utm_content=blog_article.
Search in Google Scholar Back to article
BIBER, Douglas – CONRAD, Susan (2009): Register, Genre, and Style. Cambridge: Cambridge University Press. 344 p.
Search in Google Scholar Back to article
CAFFREY, Colm – VALENTINI, Cristina (2020): Technology applications at WIPO. In: M. O´Hagan (ed.): The Routledge Handbook of Translation and Technology. New York: Routledge – Taylor and Francis Group, pp. 127–147.
Search in Google Scholar Back to article
CASTILHO, Sheila – MOORKENS, Joss – GASPERI, Frederica – CALIXTO, Iacer – TINSELY, John – WAY, Andy (2017): Is neural machine translation the new state of the art? In: The Prague Bulletin of Mathematical Linguistics, Vol. 108, No. 1, pp. 109–120.
Search in Google Scholar Back to article
DIÑO, Gino (2018): Korean Voice Assistant Highlights Tech´s Insatiable Hunger for Language Data. Slator. Available at: https://slator.com/technology/korean-voice-assistant--highlights-techs-insatiable-hunger-for-language-data/
Search in Google Scholar Back to article
DOLNÍK, Juraj (2013): Všeobecná jazykoveda. Opis a vysvetľovanie jazyka. Bratislava: Veda, vydavateľstvo SAV. 375 p.
Search in Google Scholar Back to article
DUŠKOVÁ, Libuše (2012): Mluvnice současné angličtiny na pozadí češtiny. Praha: Academia. 676 p.
Search in Google Scholar Back to article
FINDRA, Ján (2013): Štylistika súčasnej slovenčiny. Martin: Osveta. 320 p.
Search in Google Scholar Back to article
FONT LLITJOS, Ariadna – CARBONELL, G. Jaime – LAVIE, Alon (2005): A Framework for Interactive and Automatic Refinement of Transfer-based Machine Translation. In: Proceedings of the 10th EAMT Conference: Practical applications of machine translation. Budapest, Hungary: European Association for MT, pp. 87–96.
Search in Google Scholar Back to article
HLAVATÁ, Renáta – KRALČÁK, Ľubomír – GLOVŇA, Juraj – RENDÁROVÁ, Mária – DUDOVÁ, Katarína (2019): Jazyk ako text (vysokoškolský učebný materiál z praktickej štylistiky). Nitra: UKF. 174 p.
Search in Google Scholar Back to article
JEAN, Sébastien – FIRAT, Orhan – CHO, Kyunghyun – MEMISEVIC, Roland – BENGIO, Yoshua (2015): Montreal Neural Machine Translation Systems for WMT’15. In: O. Bojar – R. Chatterjee – Ch. Federmann – B. Haddow – Ch. Hokamp – M. Huck – V. Logacheva – P. Pecina (eds.): Proceedings of the 10th Workshop on Statistical Machine Translation. September 17–18, 2015 Lisboa. Portugal, Lisbon: Association for Computational Linguistics, pp. 134–140.
Search in Google Scholar Back to article
KOPONEN, Maarit (2016): Is machine translation post-editing worth the effort? A survey of research into post-editing and effort. In: The Journal of Specialised Translation, Vol. 25, pp. 131–148.
Search in Google Scholar Back to article
KVETKO, Pavol (2021): Translation Studies. Trnava: UCM. 170 p.
Search in Google Scholar Back to article
LOMMEL, Arle (2018): Metrics for Translation Quality Assessment: A Case for Standardising Error Typologies. In: J. Moorkens – S. Castilho – F. Gaspari – S. Doherty (eds): Translation Quality Assessment. Machine Translation: Technologies and Applications. Springer: Cham, pp. 109–127.
Search in Google Scholar Back to article
LUONG, Thang – SUTSKEVER, Ilya – LE, Quoc – VINYALS, Oriol – ZAREMBA, Wojciech (2015): Addressing the rare word problem in neural machine translation. In: Ch. Zong – M. Strube (eds.): Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Beijing, China: Association for CL, Vol. 1, pp. 11–19.
Search in Google Scholar Back to article
MARTÍN, Juan Alberto Alonso – SERRA, Anna Civil (2014): Integration of a machine translation system into the editorial process flow of a daily newspaper. In: Procesamiento del Lenguaje Natural, Vol. 53, pp. 193–196.
Search in Google Scholar Back to article
MELBY, K. Alan (2020): Future of machine translation. Musings on Weaver´s memo. In: M. O´Hagan (ed.): The Routledge Handbook of Translation and Technology. New York: Routledge – Taylor and Francis Group, pp. 419–436.
Search in Google Scholar Back to article
MISTRÍK, Jozef (1989): Štylistika. Bratislava: SPN.
Search in Google Scholar Back to article
MOORKENS, Joss – LEWIS, Dave (2020): Copyright and re-use of data. In: M. O´Hagan (ed.): The Routledge Handbook of Translation and Technology. New York: Routledge – Taylor and Francis Group, pp. 469–481.
Search in Google Scholar Back to article
MUNKOVÁ, Daša – MUNK, Michal (2016): Evalvácia strojového prekladu. Nitra: UKF. 173 p.
Search in Google Scholar Back to article
MUNKOVÁ, Daša (2017): Prekladateľské technológie a translatológia. In: D. Munková – J. Vaňko – J. Absolon – T. Bánik – Ľ. Benko – R. Machová – M. Munk – P. Petráš – K. Welnitzová (eds): Mýliť sa je ľudské (ale aj strojové). Nitra: UKF, pp. 15–36.
Search in Google Scholar Back to article
ONDRUŠ, Šimon – SABOL, Ján (1984): Úvod do štúdia jazykov. Bratislava: SPN. 344 p.
Search in Google Scholar Back to article
ORAVEC, Ján – BAJZÍKOVÁ, Eugénia – FURDÍK, Juraj (1984): Súčasný slovenský spisovný jazyk. Morfológia. Bratislava: SPN. 232 p.
Search in Google Scholar Back to article
PETRÁŠ, Patrik – MUNKOVÁ, Daša (2023): Strojový preklad založený na neurónových sieťach – sľubná cesta prekladu z analytických jazykov do flektívnej slovenčiny? In: Slovenská reč, Vol. 88, No. 1, pp. 74–89.
Search in Google Scholar Back to article
PLITT, Mirko – MASSELOT, Francois (2010): A productivity test of statistical machine translation post-editing in a typical localisation context. In: The Prague Bulletin of Mathematical Linguistics, Vol. 93, pp. 7–16.
Search in Google Scholar Back to article
PONDELÍKOVÁ, Ivana – LUPRICHOVÁ, Jana (2024): AI-assisted enhancing of gender awareness through reading comprehension in history and literature courses of anglophone cultures. In: G. D. Sampso – D. Ifenthaler – P. Isaias (eds): CELDA 2024: 21st International Conference on Cognition and Exploratory Learning in Digital Age: Proceedings, pp. 198–208.
Search in Google Scholar Back to article
QUAH, Chiew Kin (2006): Translation and Technology. London: Palgrave Macmillan. 256 p.
Search in Google Scholar Back to article
SLANČOVÁ, Daniela – SPÁČILOVÁ, Stanislava (2022): Štylistické výskumy v časopise Slovenská reč. In: Slovenská reč, Vol. 87, No. 3, pp. 283–306.
Search in Google Scholar Back to article
TORAL, Antonio – SÁNCHEZ-CARTAGENA, Víctor (2017): A multifaceted evaluation of neural versus phrase-based machine translation for 9 language directions. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Vol. 1, pp. 1063–1073.
Search in Google Scholar Back to article
VAJIČKOVÁ, Mária (2023): Plusquamperfekt/Antepräteritum im Deutschen und im Slowakischen als Kohäsions- und Kohärenzmittel. (Pluperfect/Antepreterite in German and Slovak as a Means of Cohesion and Coherence). In: Philologia, Vol. 33, No. 2, pp. 7–22.
Search in Google Scholar Back to article
VAŇKO, Juraj – AUXOVÁ, Darina (2015): Morfológia slovenského jazyka. Nitra: UKF. 158 p.
Search in Google Scholar Back to article
VAŇKO, Juraj (2015): Syntax slovenského jazyka. Nitra: UKF. 150 p.
Search in Google Scholar Back to article
VAŇKO, Juraj (2017): Kategoriálny rámec pre analýzu chýb strojového prekladu. In: D. Munková – J. Vaňko – J. Absolon – T. Bánik – Ľ. Benko – R. Machová – M. Munk – P. Petráš – K. Welnitzová (eds.): Mýliť sa je ľudské (ale aj strojové). Nitra: UKF, pp. 83–100.
Search in Google Scholar Back to article
VILAR, David – XU, Jia – D’HARO, Luis Fernando – NEY, Hermann (2006): Error Analysis of Statistical Machine Translation Output, In: Proceedings of LREC2006. May 22–28, 2006, Genoa, Italy. Genoa: European Language Resources Association, pp. 697–702.
Search in Google Scholar Back to article
WANG, Haifeng – WU, Hua – HE Zhongjun – HUANG, Liang – CHURCH Ward, Kenneth (2022): Progress in Machine Translation. In: Engineering, Vol. 18, pp. 143–153.
Search in Google Scholar Back to article
WELNITZOVÁ, Katarína – MUNKOVÁ, Daša (2021): Sentence-structure errors of machine translation into Slovak. In: Topics in Linguistics, Vol. 22, No. 1, pp. 78–92.
Search in Google Scholar Back to article
WELNITZOVÁ, Katarína – MUNKOVÁ, Daša – WREDE, Oľga (2020): Chybovosť v predikatívnosti a kvalita strojového prekladu. In: Jazyk a kultúra, Vol. 11, No. 41–42, pp. 160–172.
Search in Google Scholar Back to article
WELNITZOVÁ, Katarína (2023): Chybovosť strojového prekladu. Praha: Verbum. 130 p. WELNITZOVÁ, Katarína (2024): Machine Translation and its Limits. In: Acta Anglica Tyrnaviensia, Vol. 1, No. 2, pp. 22–33.
Search in Google Scholar Back to article

Persisting Grammatical Errors of Machine Translation

Abstract

Paradigm

My account