Application of Natural Language Processing Algorithms to the Task of Automatic Classification of Russian Scientific Texts

Aleksandr Romanov; Konstantin Lomotin; Ekaterina Kozlova

doi:10.5334/dsj-2019-037

Abstract

This work is devoted to the study of applicability of modern methods of machine learning to the task of automatic classification of scientific articles and abstracts. For this purpose, the study of such models of machine learning as artificial neural networks, random forest, logistic regression, and support vector machine was carried out with taking into account such a feature of scientific texts as a large number of terms specific for various categories. Separately, the stages of data collection and extraction of text characteristics are considered. The results of research are used in development of a decision support system for assignment of scientific texts to the code of the department or abstract journal of All-Russian Institute of Scientific and Technical Information of Russian Academy of Sciences.

References

Abuhaiba, ISI and Dawoud, HM. 2017. Combining Different Approaches to Improve Arabic Text Documents Classification. International Journal of Intelligent Systems and Applications, 9(4): 39–52. DOI: 10.5815/ijisa.2017.04.05
Open DOI Search in Google Scholar Back to article
Automatic Content Classification with ABBYY Solutions. 2019. Available at https://www.abbyy.com/solutions/document-classification/ [Last accessed 28.08.2018].
Search in Google Scholar Back to article
Bourgonje, P, Moreno-Schneider, J, Srivastava, A and Rehm, G. 2018. Automatic Classification of Abusive Language and Personal Attacks in Various Forms of Online Communication. 180–191. DOI: 10.1007/978-3-319-73706-5_15
Open DOI Search in Google Scholar Back to article
Chen, G, Ye, D, Xing, Z, Chen, J and Cambria, E. 2017. Ensemble application of convolutional and recurrent neural networks for multi-label text categorization. B: 2017 International Joint Conference on Neural Networks (IJCNN), 2377–2383. IEEE. DOI: 10.1109/IJCNN.2017.7966144
Open DOI Search in Google Scholar Back to article
Clark, J, Koprinska, I and Poon, J. 2003. A neural network based approach to automated e-mail classification. B: Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003), 702–705. IEEE Comput. Soc. DOI: 10.1109/WI.2003.1241300
Open DOI Search in Google Scholar Back to article
Dimov, D, Low, F, Ibrakhimov, M, Stulina, G and Conrad, C. 2017. SAR and optical time series for crop classification. B: 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 811–814. IEEE. DOI: 10.1109/IGARSS.2017.8127076
Open DOI Search in Google Scholar Back to article
Du, J. 2017. Automatic text classification algorithm based on Gauss improved convolutional neural network. Journal of Computational Science, 21: 195–200. DOI: 10.1016/j.jocs.2017.06.010
Open DOI Search in Google Scholar Back to article
Feng, K-Y, Cai, Y-D and Chou, K-C. 2005. Boosting classifier for predicting protein domain structural class. Biochemical and Biophysical Research Communications, 334(1): 213–217. DOI: 10.1016/j.bbrc.2005.06.075
Open DOI Search in Google Scholar Back to article
gensim: models.word2vec – Word2vec embeddings. 2019. Available at https://radimrehurek.com/gensim/models/word2vec [Last accessed 28.08.2018].
Search in Google Scholar Back to article
Géron, A. 2018. Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow: concepts, tools, and techniques to build intelligent systems. 1-e and. O’Reilly.
Search in Google Scholar Back to article
Goncalves, T and Quaresma, P. 2018. Evaluating preprocessing techniques in a Text Classification problem.
Search in Google Scholar Back to article
Joulin, A, Grave, E, Bojanowski, P and Mikolov, T. 2016. Bag of Tricks for Efficient Text Classification. Available at http://arxiv.org/abs/1607.01759. DOI: 10.18653/v1/E17-2068
Open DOI Search in Google Scholar Back to article
Klyshinsky, ES and Kochetkova, NA. 2015. Method of extracting technical terms using the measure of strangeness. Novye Informacionnye Tekhnologii v Avtomatizirovannyh Sistemah, 17: 365–370.
Search in Google Scholar Back to article
Korenius, T, Laurikkala, J, Järvelin, K and Juhola, M. 2004. Stemming and lemmatization in the clustering of finnish text documents. B: Proceedings of the Thirteenth ACM conference on Information and knowledge management – CIKM ’04, 625. New York, USA: ACM Press. DOI: 10.1145/1031171.1031285
Open DOI Search in Google Scholar Back to article
Kusner, M, Sun, Y, Kolkin, N and Weinberger, K. 2015. From Word Embeddings To Document Distances. Bach, F and Blei, D (ed.), B: Proceedings of the 32nd International Conference on Machine Learning, 957–966. Lille, France: PMLR. Available at http://proceedings.mlr.press/v37/kusnerb15.html.
Search in Google Scholar Back to article
Lapach, SN and Radchenko, SG. 2012. The main problems of constructing regression models. Matematicheskie Mashiny I Sistemy, 1(4): 125–133.
Search in Google Scholar Back to article
Lex, E, Seifert, C, Granitzer, M and Juffinger, A. 2010. Efficient Cross-Domain Classification of Weblogs. International Journal of Intelligent Computing Research, 1(3): 55–62. DOI: 10.20533/ijicr.2042.4655.2010.0007
Open DOI Search in Google Scholar Back to article
Liu, Y, Bi, J-W and Fan, Z-P. 2017. A method for multi-class sentiment classification based on an improved one-vs-one (OVO) strategy and the support vector machine (SVM) algorithm. Information Sciences, 394–395: 38–52. DOI: 10.1016/j.ins.2017.02.016
Open DOI Search in Google Scholar Back to article
LPU download page. 2019. Available at https://www.cs.uic.edu/~liub/LPU/LPU-download.html [Last accessed 28.08.2018].
Search in Google Scholar Back to article
Luo, Y, Ye, W, Zhao, X, Pan, X and Cao, Y. 2017. Classification of Data from Electronic Nose Using Gradient Tree Boosting Algorithm. Sensors, 17(10): 2376. DOI: 10.3390/s17102376
Open DOI Search in Google Scholar Back to article
Raju, MK, Subrahmanian, ST and Sivakumar, T. 2017. A Comparative Survey on Different Text Categorization Techniques. Journal of Computer Science and Engineering, 5(4): 1612–1618.
Search in Google Scholar Back to article
Sammouda, R. 2017. A Comparative Study of Effective Supervised Learning Methods on Arabic Text Classification, 17(12): 130–133.
Search in Google Scholar Back to article
Scherer, D, Müller, A and Behnke, S. 2010. Evaluation of pooling operations in convolutional architectures for object recognition. B: Proceeding ICANN’10 Proceedings of the 20th international conference on Artificial neural networks: Part III, 92–101. DOI: 10.1007/978-3-642-15825-4_10
Open DOI Search in Google Scholar Back to article
Semberecki, P and Maciejewski, H. 2017. Deep Learning methods for Subject Text Classification of Articles. cc. 357–360. DOI: 10.15439/2017F414
Open DOI Search in Google Scholar Back to article
Sokolova, M and Bobicev, V. 2009. Classification of Emotion Words in Russian and Romanian Languages, 416–420.
Search in Google Scholar Back to article
Sun, Y, Kamel, MS, Wong, AKC and Wang, Y. 2007. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognition, 40(12): 3358–3378. DOI: 10.1016/j.patcog.2007.04.009
Open DOI Search in Google Scholar Back to article
Tang, D, Qin, B and Liu, T. 2015. Document modeling with gated recurrent neural network for sentiment classification. B: Proceedings of the 2015 conference on empirical methods in natural language processing. cc. 1422–1432. DOI: 10.18653/v1/D15-1167
Open DOI Search in Google Scholar Back to article
Toloşi, L and Lengauer, T. 2011. Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics, 27(14): 1986–1994. DOI: 10.1093/bioinformatics/btr300
Open DOI Search in Google Scholar Back to article
Toman, M, Tesar, R and Jezek, K. 2006. Influence of word normalization on text classification. B: Proceedings of InSciT. cc. 354–358. Available at http://www.kiv.zcu.cz/research/groups/text/publications/inscit20060710.pdf.
Search in Google Scholar Back to article
Uysal, AK and Gunal, S. 2014. The impact of preprocessing on text classification. Information Processing & Management, 50(1): 104–112. DOI: 10.1016/j.ipm.2013.08.006
Open DOI Search in Google Scholar Back to article
Viniti.ru. 2019. Available at http://www.viniti.ru/ [Last accessed 01.01.2019].
Search in Google Scholar Back to article
Vorobyov, NV and Puchkov, EV. 2017. Text classification using convolutional neural network. Yuniy Issledovatel Dona, 9(6): 2–7.
Search in Google Scholar Back to article
Wang, X, Jiang, W and Luo, Z. 2016. Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts. B: Proceedings of {COLING} 2016, the 26th International Conference on Computational Linguistics: Technical Papers. Osaka, Japan: The COLING 2016 Organizing Committee. cc. 2428–2437. Available at https://www.aclweb.org/anthology/C16-1229
Search in Google Scholar Back to article
Wang, Y, Zhou, Z, Jin, S, Liu, D and Lu, M. 2017. Comparisons and Selections of Features and Classifiers for Short Text Classification. IOP Conference Series: Materials Science and Engineering, 261: 012018. DOI: 10.1088/1757-899X/261/1/012018
Open DOI Search in Google Scholar Back to article
Watson Natural Language Classifier. 2019. Available at https://www.ibm.com/watson/services/natural-language-classifier/ [Last accessed 28.08.2018].
Search in Google Scholar Back to article
Wu, L, Yen, IE-H, Xu, K, Xu, F, Balakrishnan, A, Chen, P-Y, Ravikumar, P and Witbrock, MJ. 2018. Word Mover’s Embedding: From Word2Vec to Document Embedding. CoRR, abs/1811.01713. Available at http://arxiv.org/abs/1811.01713. DOI: 10.18653/v1/D18-1482
Open DOI Search in Google Scholar Back to article
Xu, B and Zhang, Y. 2011. A new SVM Chinese text of classification algorithm based on the semantic kernel. B: 2011 International Conference on Multimedia Technology, 2857–2860. IEEE. DOI: 10.1109/ICMT.2011.6003097
Open DOI Search in Google Scholar Back to article
Yussupova, NI, Bogdanova, D and Boyko, MN. 2012. Applying of Sentiment Analysis for Texts in Russian Based on Machine Learning Approach.
Search in Google Scholar Back to article

Application of Natural Language Processing Algorithms to the Task of Automatic Classification of Russian Scientific Texts

Abstract

Paradigm

My account