Text Vectorization Techniques Based on Wordnet

Držík, Dávid; Šteflovič, Kirsten

doi:10.2478/jazcas-2023-0048

Abstract

The utilization of text vectorization techniques has become essential for numerous classification tasks in present-day natural language processing. Word embedding methods commonly used today, such as Word2Vec, GloVe, etc., are based on the semantic similarity of words. WordNet, as a lexical database of words, provides a rich source of semantic information. In our article, we propose a text vectorization technique using extended text data with the data augmentation method, specifically by replacing words with their synonyms obtained from WordNet. The results obtained from text classification tasks using multiple classifiers demonstrate that expanding the corpus with this method leads to improved vector representations of words.

References

Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit, 1st ed. Beijing; Cambridge Mass. O’Reilly Media.
Search in Google Scholar Back to article
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv, May 24, 2019. Accessed: Mar. 26, 2023. [Online]. Accessible at: http://arxiv.org/abs/1810.04805.
Search in Google Scholar Back to article
Glenski, M., Sealy, W. I., Miller, K., and Arendt, D. (2021). Improving Synonym Recommendation Using Sentence Context. In Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing, pages 74–78. Accessible at: doi 10.18653/v1/2021.sustainlp-1.9.
Search in Google Scholar Back to article
Haralabopoulos, G., Torres, M. T., Anagnostopoulos, I., and McAuley, D. (2021). Text data augmentations: Permutation, antonyms and negation. Expert Systems with Applications, Vol. 177. Accessible at: doi 10.1016/j.eswa.2021.114769.
Search in Google Scholar Back to article
Kobayashi, S. (2018). Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 452–457. New Orleans, Louisiana. Accessible at: doi 10.18653/v1/N18-2072.
Search in Google Scholar Back to article
Marivate, V., and Sefara, T. (20202). Improving Short Text Classification Through Global Augmentation Methods. In Machine Learning and Knowledge Extraction, pages 385–399. Accessible at: doi 10.1007/978-3-030-57321-8_21.
Search in Google Scholar Back to article
Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. ArXiv. Accessible at: doi 10.48550/arXiv.1301.3781.
Search in Google Scholar Back to article
Miller, G. A. (1995). WordNet: A lexical database for English, Commun. ACM, 38(11), pages 39–41. Accessible at: doi 10.1145/219717.219748.
Search in Google Scholar Back to article
Nazir, S, Asif, M., Sahi, S. A., Ahmad, S., Ghadi, Y. Y., and Aziz, M. H. (2022). Toward the Development of Large-Scale Word Embedding for Low-Resourced Language. IEEE Access, Vol. 10, pages 54091–54097. Accessible at: doi 10.1109/ACCESS.2022.3173259.
Search in Google Scholar Back to article
Pellicer, L. F. A. O., Ferreira, T. M., and Costa, A. H. R. (2023). Data augmentation techniques in natural language processing. Applied Soft Computing, Vol. 132. Accessible at: doi 10.1016/j.asoc.2022.109803.
Search in Google Scholar Back to article
Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Doha, Qatar. Accessible at: doi 10.3115/v1/D14-1162.
Search in Google Scholar Back to article
Risdal, M. (2016). Getting Real about Fake News, Kaggle. Accessible at: doi 10.34740/KAGGLE/DSV/911.
Search in Google Scholar Back to article
Rossum, G. V. (1999). Python Library Reference. To Excel Inc.
Search in Google Scholar Back to article
Řehurek, R., and Sojka, P. (2010). Software Framework for Topic Modelling with Large Corpora, pages 45–50. Accessible at: doi 10.13140/2.1.2393.1847.
Search in Google Scholar Back to article
Shi, Y., Zheng, Y., Guo, K., Zhu, L., and Qu, Y. (2018). Intrinsic or Extrinsic Evaluation: An Overview of Word Embedding Evaluation. In 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pages 1255–1262. Singapore. Accessible at: doi 10.1109/ICDMW.2018.00179.
Search in Google Scholar Back to article
Verma, P. K., Agrawal, P., Amorim, I., and Prodan, R. (2021). WELFake: Word Embedding Over Linguistic Features for Fake News Detection. IEEE Trans. Comput. Soc. Syst., 8(4), pages 881–893. Accessible at: doi 10.1109/TCSS.2021.3068519.
Search in Google Scholar Back to article
Wang, W. Y., and Yang, D. (2015). That’s So Annoying‼!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2557–2563. Lisbon, Portugal. Accessible at: doi 10.18653/v1/D15-1306.
Search in Google Scholar Back to article
Wei, J., and Zou, K. (2019). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. arXiv, Aug. 25, 2019. Accessible at: doi 10.48550/arXiv.1901.11196.
Search in Google Scholar Back to article
Zhai, M., Tan, J., and Choi, J. (2016). Intrinsic and Extrinsic Evaluations of Word Embeddings. AAAI, 30(1). Accessible at: doi 10.1609/aaai.v30i1.9959.
Search in Google Scholar Back to article