AI and the Translation of Idioms – Challenges, Success, and a Corpus Perspective

Filip Kalaš; Pavol Lipták

doi:10.2478/jazcas-2025-0023

Abstract

This study explores the capabilities of artificial intelligence in translating English idioms into Slovak, with and without contextual information. Using a dataset of 100 idioms and evaluating AI-generated translations against a validated bilingual dictionary of idioms, both qualitative and statistical analyses were employed. The results show an unexpectedly high accuracy in context-free translations, while context occasionally led to deterioration. McNemar’s test and a t-test confirmed a statistically significant shift in performance. The study highlights key advantages and limitations of AI, suggesting further research into reverse and cross-linguistic translation as well as employment of corpus-based methods.

References

Abjalova, M., and Sharipova, S. (2024). Semantic and Grammatical Issues in Translating Idioms with Automatic Translation Systems. In 2024 9^th International Conference on Computer Science and Engineering, pp. 58–63.
Search in Google Scholar Back to article
Artamonova, M. V. et al. (2023). Chatbot as a translation tool. In Litera 8, pp. 235–253.
Search in Google Scholar Back to article
Baziotis, Ch. et al. (2023). Automatic Evaluation and Analysis of Idioms in Neural Machine Translation. In Proceedings of the 17^th Conference of the European Chapter of the Association for Computational Linguistics, Dubrovnik: Association for Computational Linguistics, pp. 3682–3700. Accessible at: https://aclanthology.org/2023.eacl-main.267/.
Search in Google Scholar Back to article
Derner, E., and Batistič, K. (2023). Beyond the Safeguards: Exploring the Security Risks of ChatGPT. Accessible at: https://arxiv.org/abs/2305.08005.
Search in Google Scholar Back to article
Dong, Y. et al. (2019): Acquisition of interpreting strategies by student interpreters. The Interpreter and Translator Trainer 13(4), pp. 408–425.
Search in Google Scholar Back to article
Donthi, S. (2025). Improving LLM Abilities in Idiomatic Translation. In Proceedings of the First Workshop on Language Models for Low-Resource Languages. Abu Dhabi: Association for Computational Linguistics, pp. 175–181. Accessible at: https://arxiv.org/abs/2407.03518.
Search in Google Scholar Back to article
Hakami, A. H., and Abomoati, G. S. (2024). Exploring the Impact of Prompt Formulation in AI Chatbots on the Translation of English-to-Arabic and Arabic-to-English Idioms: A Case-Study. Pakistan Journal of Life and Social Sciences 22(2), pp. 21371–21381.
Search in Google Scholar Back to article
Hamood, M. I. (2024). The Translation of English Food Idioms into Arabic Through ChatGPT: Problems and Solutions. Accessible at: https://shorturl.at/9TcUT.
Search in Google Scholar Back to article
Jarošová, A. et al. (2021). Slovník súčasného slovenského jazyka. Bratislava: Veda.
Search in Google Scholar Back to article
Jiao, W. et al. (2023). Is ChatGPT a Good Translator? Yes With GPT-4 As The Engine. Accessible at: https://arxiv.org/abs/2301.08745.
Search in Google Scholar Back to article
Li, J. et al. (2024). Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions. In Transactions of the Association for Computational Linguistics 12, pp. 576–592. Accessible at: https://aclanthology.org/2024.tacl-1.32/.
Search in Google Scholar Back to article
Li, S. et al. (2023). Translate Meanings, Not Just Words: IdiomKB’s Role in Optimizing Idiomatic Translation with Language Models. In Computation and Language. Accessible at: https://arxiv.org/abs/2308.13961.
Search in Google Scholar Back to article
Lund, B. (2023). ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing. In Journal of the Association for Information Science and Technology. Accessible at: https://arxiv.org/abs/2303.13367.
Search in Google Scholar Back to article
Mughal, U. A. et al. (2024). The intersection of linguistics and artificial intelligence: A corpus-based study of idiom translation. Journal of applied linguistics and Tesol 7(4), pp. 1453–1460.
Search in Google Scholar Back to article
Obeidat, M. M. et al. (2024). Analyzing the Performance of Gemini, ChatGPT, and Google Translate into Rendering English Idioms into Arabic. Journal of Social Sciences 18(4), pp. 1–18.
Search in Google Scholar Back to article
Rodriguez, P. R. (2024). Phraseological evaluation of automatic interpretation assisted by Yandex. Translation Matters 6(2), pp. 115–130.
Search in Google Scholar Back to article
Sinclair, J. (1991). Corpus, Concordance, Collocation. Oxford: Oxford University Press. Sison, A. J. et al. (2023). ChatGPT: More than a Weapon of Mass Deception, Ethical challenges and responses from the Human-Centered Artificial Intelligence (HCAI) perspective. Accessible at: https://arxiv.org/abs/2304.11215.
Search in Google Scholar Back to article
Wang, X., and Fantinuoli, C. (2024). Exploring the correlation between human and machine evaluation of simultaneous speech translation. In Proceedings of the 25^th Annual Conference of the European Association for Machine Translation, Sheffield: EAMT, pp. 327–336. Accessible at: https://aclanthology.org/2024.eamt-1.28/.
Search in Google Scholar Back to article
Zhu, W. (2024). Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. In Findings of the Association for Computational Linguistics: NAACL 2024. Mexico City: Association for Computational Linguistics, pp. 2765–2781. Accessible at: https://arxiv.org/abs/2304.04675.
Search in Google Scholar Back to article

AI and the Translation of Idioms – Challenges, Success, and a Corpus Perspective

Abstract

Paradigm

My account