Sentiment analysis of cultural differences in online comments on popular news

Hordiienko, Kateryna; Kormaníková, Libuše

doi:10.2478/lf-2025-0020

Abstract

The rapid growth of online communication through social networks has created new opportunities for understanding public opinion on socially relevant issues. This research examines how sentiment analysis (SA) can reveal cultural differences, specifically analyzing Czech and Ukrainian online comments on news topics including the war in Ukraine, political discussions, public health issues (tick-borne diseases, COVID-19), LGBTQ+ community matters, and natural disasters. By comparing three large language models (GPT-3.5-Turbo, Twitter-XLM-Ro-BERTa, and Zephyr 7B) with native speaker evaluations, we assess whether AI-based sentiment analysis can accurately capture culturally-specific emotional expressions in medium-resource languages. Our dataset comprises 6,085 comments (2,999 Czech from X/Twitter, 3,086 Ukrainian from Telegram) collected during 2023, focusing on socially relevant news coverage. We employed a hybrid methodology combining machine learning analysis with expert validation by native speakers. The study addresses a critical gap in cross-cultural sentiment analysis research, as no previous studies have compared Czech and Ukrainian linguistic patterns in this context. Results demonstrate significant performance differences among models depending on language: GPT-3.5-Turbo achieved highest accuracy for Czech (p<0.001), while all models performed comparably for Ukrainian. Both populations showed predominantly negative sentiment (Czech: 69.93%, Ukrainian: 68.93% via GPT-3.5), reflecting shared emotional responses to crisis events.

References

Amirizaniani, M., Martin, E., Sivachenko, M. et al., 2024, October. Can LLMs reason like humans? Assessing theory of mind reasoning in LLMs for open-ended questions. In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pp. 34–44.
Search in Google Scholar Back to article
Anderwald, L., 2018. Language change and cultural change: The grammaticalization of the GET-passive in context. Language & Communication, 62, 1–14. Aslan, S., 2023. A deep learning-based sentiment analysis approach (MF-CNN-BILSTM) and topic modeling of tweets related to the Ukraine–Russia conflict. Applied Soft Computing, 143, 110404, available at: https://doi.org/10.1016/j.asoc.2023.110404.
Search in Google Scholar Back to article
Barbieri, F., Espinosa Anke, L., Camacho-Collados, J., 2022. XLM-T: Multilingual language models in Twitter for sentiment analysis and beyond. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference, European Language Resources Association, pp. 258–266, available at: https://aclanthology.org/2022.lrec-1.27.
Search in Google Scholar Back to article
Boroditsky, L., 2011. How language shapes thought. Scientific American, 304(2), 62–65.
Search in Google Scholar Back to article
Boyd, D. M., Ellison, N. B., 2007. Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), 210–230, available at: https://doi.org/10.1111/j.1083-6101.2007.00393.x.
Search in Google Scholar Back to article
Bose, R., Dey, R. K., Roy, S., Sarddar, D., 2020. Sentiment analysis on online product reviews. In: Tuba, M., Akashe, S., Joshi, A. (Eds.), Information and Communication Technology for Sustainable Development (Advances in Intelligent Systems and Computing, Vol. 933). Singapore: Springer, pp. 559–569, available at: https://doi.org/10.1007/978-981-13-7166-0_56.
Search in Google Scholar Back to article
Dahlberg, G. M., Bagga-Gupta, S., 2014. Understanding global learning spaces: An empirical study of languaging and transmigrant positions in the virtual classroom. Learning, Media and Technology, 39(4), 468–487, available at: https://doi.org/10.1080/17439884.2014.931868.
Search in Google Scholar Back to article
Ellison, N., Steinfield, C., Lampe, C., 2007. The benefits of Facebook “friends”: Exploring the relationship between college students’ use of online social networks and social capital. Journal of Computer-Mediated Communication, 12(3), article 1, available at: http://jcmc.indiana.edu/vol12/issue4/ellison.html.
Search in Google Scholar Back to article
Field, J., 2004. Psycholinguistics: The key concepts. Routledge.
Search in Google Scholar Back to article
Flek, L., 2020. Returning the N to NLP: Towards contextually personalized classification models. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7828– 7838, available at: https://doi.org/10.18653/v1/2020.acl-main.700.
Search in Google Scholar Back to article
Havaldar, S., Singhal, B., Rai, S. et al., 2023, July. Multilingual language models are not multicultural: A case study in emotion. In: Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, Association for Computational Linguistics, pp. 202–214, available at: https://doi.org/10.18653/v1/2023.wassa-1.19.
Search in Google Scholar Back to article
Hassonah, M. A., Al-Sayyed, R., Rodan, A. et al., 2020. An efficient hybrid filter and evolutionary wrapper approach for sentiment analysis of various topics on Twitter. Knowledge-Based Systems, 192, 105353, available at: https://doi.org/10.1016/j.knosys.2019.105353.
Search in Google Scholar Back to article
Hordiienko, K., 2024. AI and human sentiment analysis with cultural aspect. Poster presented at the 21st International Congress of Linguists, Poznań, Poland. Hordiienko, K., 2021. Moral disengagement as a factor of cyberbullying among students. Habitus, (29), 147–151, retrieved from: http://nbuv.gov.ua/UJRN/habit_2021_29_27.
Search in Google Scholar Back to article
Hordiienko, K., Joukl, Z., 2023, August 11–20. Sentiment analysis of different nationalities’ internet comments on extraordinary news: cultural aspect. České Budějovice, Czech Republic: Summer School of Linguistics, retrieved from: https://ssol.ff.cuni.cz/summer-school-of-linguistics/ssol-2023/.
Search in Google Scholar Back to article
Hordiienko, K., Joukl, Z., 2024. Sentimental reflection of global crises: Czech and Ukrainian views on popular events through the prism of internet commentary. Jazykovedný Časopis, 75(1), 43–61, available at: https://doi.org/10.2478/jazcas-2024-0027.
Search in Google Scholar Back to article
Hung, L. P., Alias, S., 2023. Beyond sentiment analysis: A review of recent trends in text-based sentiment analysis and emotion detection. Journal of Advanced Computational Intelligence and Intelligent Informatics, 27(1), 84–95, available at: https://doi.org/10.20965/jaciii.2023.p0084.
Search in Google Scholar Back to article
Hunston, S., 2010. Corpus approaches to evaluation: Phraseology and evaluative language. Routledge.
Search in Google Scholar Back to article
Karakikes, A., Alexiadis, P., Kotis, K., 2024. Bias in X (Twitter) and Telegram-Based Intelligence Analysis: Exploring Challenges and Potential Mitigating Roles of AI. SN Computer Science, 5(5), 574, available at: https://doi.org/10.1007/s42979-024-02935-w.
Search in Google Scholar Back to article
Kheiri, K., Karimi, H., 2023. SentimentGPT: Exploiting GPT for advanced sentiment analysis and its departure from current machine learning. arXiv preprint, arXiv:2307.10234.
Search in Google Scholar Back to article
Krippendorf, K., 2004. Content analysis: An introduction to its methodology. Sage Publications. Krishnamoorthy, S., 2018. Sentiment analysis of financial news articles using performance indicators. Knowledge and Information Systems, 56(2), 373–394, available at: https://doi.org/10.48550/arXiv.1811.11008.
Search in Google Scholar Back to article
Krugmann, J. O., Hartmann, J., 2024. Sentiment analysis in the age of generative AI. Customer Needs and Solutions, 11(3), available at: https://doi.org/10.1007/s40547-024-00143-4.
Search in Google Scholar Back to article
Liu, B., 2012. Sentiment analysis and opinion mining. Morgan & Claypool Publishers, available at: https://doi.org/10.2200/S00416ED1V01Y201204HLT016.
Search in Google Scholar Back to article
Liu, B., Zhang, L., 2012. A survey of opinion mining and sentiment analysis. In: Aggarwal, C. C., Zhai, C. (Eds.), Mining text data. Springer, pp. 415–463, available at: https://doi.org/10.1007/978-1-4614-3223-4_13.
Search in Google Scholar Back to article
Mao, Y., Liu, Q., Zhang, Y., 2024. Sentiment analysis methods, applications, and challenges: A systematic literature review. Journal of King Saud University – Computer and Information Sciences, 36(4), Article 102048, available at: https://doi.org/10.1016/j.jksuci.2024.102048.
Search in Google Scholar Back to article
Martin, J. R., White, P. R., 2005. The language of evaluation, vol. 2. Basingstoke: Palgrave Macmillan.
Search in Google Scholar Back to article
Matlach, V., 2023. Úvod do zpracování dat 1 [Introduction to data processing 1]. Olomouc: Univerzita Palackého v Olomouci, VUP.
Search in Google Scholar Back to article
Medhat, W., Hassan, A., Korashy, H., 2014. Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal, 5(4), 1093–1113, available at: https://doi.org/10.1016/j.asej.2014.04.011.
Search in Google Scholar Back to article
Müngen, A. A., Aygün, İ., Kaya, M., 2020. Finding the relationship between news and social media users’ emotions in the COVID-19 process. Sakarya University Journal of Computer and Information Sciences, 3(3), 250–263, available at: https://doi.org/10.35377/saucis.03.03.830867.
Search in Google Scholar Back to article
Nemesh, O. M., 2017. Virtual activity of personality: Structure and dynamics of psychological content. Slovo.
Search in Google Scholar Back to article
Pang, B., Lee, L., 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1–2), 1–135.
Search in Google Scholar Back to article
Pavitha, N., Pungliya, V., Raut, A. et al., 2022. Movie recommendation and sentiment analysis using machine learning. Global Transitions Proceedings, 3(1), 279–284, available at: https://doi.org/10.1016/j.gltp.2022.03.012.
Search in Google Scholar Back to article
Pomytkina, L., Podkopaieva, Y., Hordiienko, K., 2021. Peculiarities of manifestation of student youth’ roles and positions in the cyberbullying process. International Journal of Modern Education and Computer Science, 13(6), 1–10, available at: https://doi.org/10.5815/ijmecs.2021.06.01.
Search in Google Scholar Back to article
Prytula, M., 2024. Fine-tuning of BERT, DistilBERT, XLM-RoBERTa, and Ukr-RoBERTa models for sentiment analysis of reviews in the Ukrainian language. Machine Learning, 3, 4.
Search in Google Scholar Back to article
Ramos, L., Chang, O., 2023. Sentiment analysis of Russia-Ukraine conflict tweets using RoBERTa. Uniciencia, 37(1), 421–431, available at: http://dx.doi.org/10.15359/r.37-1.23.
Search in Google Scholar Back to article
Rao, G., Huang, W., Feng, Z., Cong, Q., 2018. LSTM with sentence representations for document-level sentiment classification. Neurocomputing, 308, 49–57, available at: https://doi.org/10.1016/j.neucom.2018.04.045.
Search in Google Scholar Back to article
Robertson, C. E., Pröllochs, N., Schwarzenegger, K. et al., 2023. Negativity drives online news consumption. Nature Human Behaviour, 7(5), 812–822, available at: https://doi.org/10.1038/s41562-023-01538-4.
Search in Google Scholar Back to article
Rosenthal, S., Nakov, P., Kiritchenko, S. et al., 2015. SemEval-2015 Task 10: Sentiment analysis in Twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-2015), pp. 451–463.
Search in Google Scholar Back to article
Sánchez-Rada, J. F., Iglesias, C. A., 2019. Social context in sentiment analysis: Formal definition, overview of current trends and framework for comparison. Information Fusion, 52, 344–356, available at: https://doi.org/10.1016/j.inffus.2019.05.003.
Search in Google Scholar Back to article
Schouten, K., Frasincar, F., 2015. The benefit of concept-based features for sentiment analysis. In: Gandon, F., Cabrio, E., Stankovic, M., Zimmermann, A. (Eds.), Semantic Web Evaluation Challenges: SemWebEval 2015. Springer, vol. 548, pp. 273–287, available at: https://doi.org/10.1007/978-3-319-25518-7_19.
Search in Google Scholar Back to article
Shaik, Z. H., Prasanna, D., Jahnavi, E. et al., 2024, June. FeedForward at SemEval-2024 Task 10: Trigger and sentext-height enriched emotion analysis in multi-party conversations. In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pp. 745–756.
Search in Google Scholar Back to article
Sharma, P., Mishra, N., 2016. Feature level sentiment analysis on movie reviews. In: Proceedings of the 2nd International Conference on Next Generation Computing Technologies (NGCT), pp. 306–311. IEEE.
Search in Google Scholar Back to article
Shelke, N., Deshpande, S., Thakare, V., 2017. Domain independent approach for aspect-oriented sentiment analysis for product reviews. In: Proceedings of the 5th International Conference on Frontiers in Intelligent Computing: Theory and Applications: FICTA 2016, vol. 2, Springer Singapore, pp. 651–659.
Search in Google Scholar Back to article
Taboada, M., 2016. Sentiment analysis: An overview from linguistics. Annual Review of Linguistics, 2(1), 325–347, available at: https://doi.org/10.1146/annurev-linguistics-011415-040518.
Search in Google Scholar Back to article
Tan, K. L., Lee, C. P., Lim, K. M., 2023. RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis. Applied Sciences, 13(6), 3915, available at: https://doi.org/10.3390/app13063915.
Search in Google Scholar Back to article
Tao, Y., Viberg, O., Baker, R. S., Kizilcec, R. F., 2024. Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9), 346, available at: https://doi.org/10.1093/pnasnexus/pgae346.
Search in Google Scholar Back to article
Thakkar, G., Preradović, N. M., Tadić, M., 2024. Examining sentiment analysis for low-resource languages with data augmentation techniques. Eng, 5(4), 2920–2942.
Search in Google Scholar Back to article
Tunstall, L., Beeching, E., Lambert, N. et al., 2023. Zephyr: Direct distillation of LM alignment. arXiv preprint, arXiv:2310.16944.
Search in Google Scholar Back to article
Vergho, T., Godbout, J. F., Rabbany, R., Pelrine, K., 2024. Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigation. arXiv preprint, arXiv:2401.06920, available at: https://doi.org/10.48550/arXiv.2401.06920.
Search in Google Scholar Back to article
Veselovská, K., 2017. Sentiment analysis in Czech. Ústav formální a aplikované lingvistiky.
Search in Google Scholar Back to article
Vidhya, R., Gopalakrishnan, P., Vallamkondu, N. K., 2021. Sentiment analysis using machine learning classifiers: Evaluation of performance. In: Proceedings of the First International Conference on Computing, Communication and Control System (I3CAC 2021), June 07–08. Chennai, India: Bharath University, available at: https://doi.org/10.4108/eai.7-6-2021.2308565.
Search in Google Scholar Back to article
Wankhade, M., Rao, A. C., Kulkarni, C., 2022. A survey on sentiment analysis methods, applications, and challenges. Artificial Intelligence Review, 55, 5731–5780, available at: https://doi.org/10.1007/s10462-022-10144-1.
Search in Google Scholar Back to article
Xu, G., Meng, Y., Qiu, X. et al., 2019. Sentiment analysis of comment texts based on BiLSTM. IEEE Access, 7, 51522–51532, available at: https://doi.org/10.1109/ACCESS.2019.2909919.
Search in Google Scholar Back to article
Yang, T., Majó-Vázquez, S., Nielsen, R. K., González-Bailón, S., 2020. Exposure to news grows less fragmented with an increase in mobile access. Proceedings of the National Academy of Sciences of the United States of America, 117(46), 28678–28683, available at: https://doi.org/10.1073/pnas.2006089117.
Search in Google Scholar Back to article
Ye, J., Chen, X., Xu, N. et al., 2023. A comprehensive capability analysis of GPT-3 and GPT-3.5 series models. arXiv preprint, arXiv:2303.10420, available at: https://doi.org/10.48550/arXiv.2303.10420.
Search in Google Scholar Back to article
Yuna, D., Xiaokun, L., Jianing, L., Lu, H., 2022. Cross-cultural communication on social media: Review from the perspective of cultural psychology and neuroscience. Frontiers in Psychology, 13, 858900, available at: https://doi.org/10.3389/fpsyg.2022.858900.
Search in Google Scholar Back to article

Sentiment analysis of cultural differences in online comments on popular news

Abstract

Paradigm

My account