PhoBERT: Application in Disease Classification based on Vietnamese Symptom Analysis

Nguyen, Hai Thanh; Huynh, Tuyet Ngoc; Mai, Nhi Thien Ngoc; Le, Khoa Dang Dang; Thi-Ngoc-Diem, Pham

doi:10.2478/acss-2023-0004

Abstract

Besides the successful use of support software in cutting-edge medical procedures, the significance of determining a disease early signs and symptoms before its detection is a growing pressing requirement to raise the standard of medical examination and treatment. This creates favourable conditions, reduces patient inconvenience and hospital overcrowding. Before transferring patients to an appropriate doctor, healthcare staff must have the patient’s symptoms. This study leverages the PhoBERT model to assist in classifying patients with text classification tasks based on symptoms they provided in the first stages of Vietnamese hospital admission. The outcomes of PhoBERT on more than 200 000 text-based symptoms collected from Vietnamese hospitals can improve the classification performance compared to Bag of Words (BOW) with classic machine learning algorithms, and some considered deep learning architectures such as 1D-Convolutional Neural Networks and Long Short-Term Memory. The proposed method can achieve promising results to be deployed in automatic hospital admission procedures in Vietnam.

References

S. Thi Thao Nguyen, E. Yamamoto, M. Thi Ngoc Nguyen, H. Bao Le, T. Kariya, Y. M. Saw, C. Duc Nguyen, and N. Hamajima, “Waiting time in the outpatient clinic at a national hospital in Vietnam,” Nagoya J. Med. Sci., vol. 80, no. 2, pp. 227–239, May 2018.
Search in Google Scholar Back to article
D. Q. Nguyen and A. T. Nguyen, “PhoBERT: Pre-trained language models for Vietnamese,” in Findings of the Association for Computational Linguistics: EMNLP 2020, Nov. 2020, pp. 1037–1042. https://doi.org/10.18653/v1/2020.findings-emnlp.92
Search in Google Scholar Back to article
K. Kowsari, J. Meimandi, M. Heidarysafa, S. Mendu, L. Barnes, and D. Brown, “Text classification algorithms: A survey,” Information, vol. 10, no. 4, Art. no. 150, Apr. 2019. https://doi.org/10.3390/info10040150
Search in Google Scholar Back to article
V. Dogra, S. Verma, Kavita, P. Chatterjee, J. Shafi, J. Choi, and M. F. Ijaz, “A complete process of text classification system using state-of-the-art NLP models,” Computational Intelligence and Neuroscience, vol. 2022, Art. no. 1883698, Jun. 2022. https://doi.org/10.1155/2022/1883698
Search in Google Scholar Back to article
S. Chua, F. Coenen, and G. Malcolm, “Classification inductive rule learning with negated features,” in L. Cao, Y. Feng, and J. Zhong, Eds. Advanced Data Mining and Applications. ADMA 2010. Lecture Notes in Computer Science, vol. 6440. Springer, Berlin, Heidelberg, 2010, pp. 125–136. https://doi.org/10.1007/978-3-642-17316-5_12
Search in Google Scholar Back to article
M. K. Dalal and M. A. Zaveri, “Automatic text classification: a technical review,” International Journal of Computer Applications, vol. 28, no. 2, pp. 37–40, Aug. 2011. https://doi.org/10.5120/3358-4633
Search in Google Scholar Back to article
S.-B. Kim, K.-S. Han, H.-C. Rim, and S. H. Myaeng, “Some effective techniques for naive Bayes text classification,” IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 11, pp. 1457–1466, Nov. 2006. https://doi.org/10.1109/tkde.2006.180
Search in Google Scholar Back to article
M. J. Meena and K. R. Chandran, “Bayes text classification with positive features selected by statistical method,” in 2009 First International Conference on Advanced Computing, Chennai, India, Dec. 2009, pp. 28–33. https://doi.org/10.1109/icadvc.2009.5378273
Search in Google Scholar Back to article
W. Zhang, T. Yoshida, and X. Tang, “Text classification using multiword features,” in 2007 IEEE International Conference on Systems, Man and Cybernetics, Montreal, Que., Oct. 2007, pp. 3519–3524. https://doi.org/10.1109/icsmc.2007.4414208
Search in Google Scholar Back to article
L. Hao and L. Hao, “Automatic identification of stop words in Chinese text classification,” in 2008 International Conference on Computer Science and Software Engineering, Wuhan, China, Dec. 2008, pp. 718–722. https://doi.org/10.1109/csse.2008.829
Search in Google Scholar Back to article
M. Porter, “An algorithm for suffix stripping,” Program: electronic library and information systems, vol. 14, no. 3, pp. 130–137, Mar. 1980. https://doi.org/10.1108/eb046814
Search in Google Scholar Back to article
K. S. Jones, “A statistical interpretation of term specificity and its application in retrieval,” Journal of Documentation, vol. 28, no. 1, pp. 11–21, Jan. 1972. https://doi.org/10.1108/eb026526
Search in Google Scholar Back to article
W. Zhang, T. Yoshida, and X. Tang, “TFIDF, LSI and multi-word in information retrieval and text categorization,” in 2008 IEEE International Conference on Systems, Man and Cybernetics, Singapore, Oct. 2008, pp. 108–113. https://doi.org/10.1109/icsmc.2008.4811259
Search in Google Scholar Back to article
K. S. Jones, “IDF term weighting and IR research lessons,” Journal of Documentation, vol. 60, no. 5, pp. 521–523, Oct. 2004. https://doi.org/10.1108/00220410410560591
Search in Google Scholar Back to article
T. Liu, Z. Chen, B. Zhang, W. Ying Ma, and G. Wu, “Improving text classification using local latent semantic indexing,” in Fourth IEEE International Conference on Data Mining (ICDM’04), Brighton, UK, Nov. 2004, pp. 162–169. https://doi.org/10.1109/icdm.2004.10096
Search in Google Scholar Back to article
S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” Journal of the American Society for Information Science, vol. 41, no. 6, pp. 391–407, Sep. 1990. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
Search in Google Scholar Back to article
K. Church and P. Hanks, “Word association norms, mutual information, and lexicography,” Computational Linguistics, vol. 16, no. 1, pp. 22–29, Mar. 1990. https://dl.acm.org/doi/pdf/10.5555/89086.89095
Search in Google Scholar Back to article
L. Wang, F. Chu, and W. Xie, “Accurate cancer classification using expressions o f very few genes,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 4, no. 1, pp. 40–53, Jan. 2007. https://doi.org/10.1109/tcbb.2007.1006
Search in Google Scholar Back to article
R. Cohen, I. Aviram, M. Elhadad, and N. Elhadad, “Redundancy-aware topic modeling for patient record notes,” PloS One, vol. 9, no. 2, Feb. 2014, Art. no. e87555. https://doi.org/10.1371/journal.pone.0087555
Search in Google Scholar Back to article
B. J. Marafino, J. M. Davies, N. S. Bardach, M. L. Dean, and R. A. Dudley, “N-gram support vector machines for scalable procedure and diagnosis classification, with applications to clinical free text data from the intensive care unit,” Journal of the American Medical Informatics Association, vol. 21, no. 5, pp. 871–875, Apr. 2014. https://doi.org/10.1136/amiajnl-2014-002694
Search in Google Scholar Back to article
A. Tripathy, A. Anand, and S. K. Rath, “Document-level sentiment classification using hybrid machine learning approach,” Knowledge and Information Systems, vol. 53, no. 3, pp. 805–831, May 2017. https://doi.org/10.1007/s10115-017-1055-z
Search in Google Scholar Back to article
M. Cui, R. Bai, Z. Lu, X. Li, U. Aickelin, and P. Ge, “Regular expression based medical text classification using constructive heuristic approach,” IEEE Access, vol. 7, pp. 147892–147904, Oct. 2019. https://doi.org/10.1109/access.2019.2946622
Search in Google Scholar Back to article
N. Sun and C. Du, “News text classification method and simulation based on the hybrid deep learning model,” Complexity, vol. 2021, Jun. 2021, Art. no. 8064579. https://doi.org/10.1155/2021/8064579
Search in Google Scholar Back to article
K. D. D. Le, H. H. Luong, and H. T. Nguyen, “Patient classification based on symptoms using machine learning algorithms supporting hospital admission,” in P. Cong Vinh, N. Huu Nhan, Eds. Nature of Computation and Communication. ICTCC 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 408. Springer, Cham. 2021, pp. 40–50. https://doi.org/10.1007/978-3-030-92942-8_4
Search in Google Scholar Back to article
H. T. Nguyen, K. D. D. Le, N. H. Pham, and C. L. H. Tran, “Deep bidirectional LSTM for disease classification supporting hospital admission based on pre-diagnosis: a case study in Vietnam,” International Journal of Information Technology, May 2023. https://doi.org/10.1007/s41870-023-01283-x
Search in Google Scholar Back to article
A. Rios and R. Kavuluru, “Convolutional neural networks for biomedical text classification,” in BCB ‘15: Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics, Sep. 2015, pp. 258–267. https://doi.org/10.1145/2808719.2808746
Search in Google Scholar Back to article
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
Search in Google Scholar Back to article
A. Graves, “Long short-term memory,” in Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, vol 385. Springer, Berlin, Heidelberg, 2012, pp. 37–45. https://doi.org/10.1007/978-3-642-24797-2_4
Search in Google Scholar Back to article
P. Zhou, Z. Qi, S. Zheng, J. Xu, H. Bao, and B. Xu, “Text classification improved by integrating bidirectional LSTM with two-dimensional max pooling,” in Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan: The COLING 2016 Organizing Committee, Dec. 2016, pp. 3485–3495. [Online]. Available: https://aclanthology.org/C16-1329
Search in Google Scholar Back to article

PhoBERT: Application in Disease Classification based on Vietnamese Symptom Analysis

Abstract

Paradigm

My account