Feature Sorting Algorithm Based on XGBoost and MIC Combination Model

Xiang, Gao; Jun, Yu; Zhiyi, Hu; Yuzhe, Hu

doi:10.21307/ijanmc-2021-037

Abstract

Feature ranking can not only help the data analysis system improve efficiency, but also reduce the interference of redundant features and irrelevant features to the results. At present, feature ranking of massive data is an important and difficult problem. In order to solve the above problems, this paper proposes a feature importance ranking algorithm based on XGBoost and MIC model by analyzing the existing algorithm models. Firstly, XGBoost model and MIC model are established respectively; Then, the results of the above two models are weighted and combined by the error reciprocal method. XGBoost model has the advantages of high efficiency, flexibility and portability, while MIC model has universality and easy parameter adjustment. The resulting XGBoost MIC combination model has both advantages; Finally, the first mock exam is used as a sample set of data for anticancer drug candidates. After preprocessing the data set, the XGBoost-MIC combination model is used to analyze the case. At the same time, the calculation results of a single model are calculated, and the model is optimized by adjusting the parameters of the model. The results show that the error of the first mock exam is obviously lower than that of the single calculation model, and the accuracy of the XGBoost-MIC is 0.75, which is 0.02 higher than that of the single model.

References

Zhang Yu, Zhang Yansong, Chen Hong, Susan Wang. OLAP foreign key join algorithm for MIC coprocessor [J]. Journal of Software, 2017, 28(03):490–501.
Search in Google Scholar Back to article
Yang Guijun, Xu Xue, Zhao Fuqiang. User score prediction model based on XGBoost algorithm and its application [J]. Data Analysis and Knowledge Discovery, 2019, 3(01):118–126.
Search in Google Scholar Back to article
Ye Qianyi, Rao Hong, Ji Mingshu. Commercial sales forecast based on Xgboost [J]. Journal of Nanchang University (Science Edition), 2017, 41(03):275–281.
Search in Google Scholar Back to article
Chen Zhenyu, Liu Jinbo, Jerry Lee, Ji Xiaohui, Li Dapeng, Huang Yunhao, Di Fangchun, Gao Xingyu, Xu Lizhong. Ultra-short term power load forecasting based on LSTM and XGBoost combined model [J]. Power Grid Technology, 2020, 44(02):614–620.
Search in Google Scholar Back to article
Zhang Chengchang, Zhang Huayu, Luo Jianchang, He Feng. Analysis method of massive electricity consumption data based on cloud computing and improved K-means algorithm [J]. Computer Applications, 2018, 38(01):159–164.
Search in Google Scholar Back to article
Liu Nian, Liu Yu. Research on visualization technology of massive relational data based on clustering analysis algorithm [J]. Electronic Design Engineering, 2018, 26(10):92–95.
Search in Google Scholar Back to article
Cheng Xueqi, Jin Xiaolong, Wang Yuanzhuo, Guo Jiafeng, Zhang Tieying, Li Guojie. Overview of Big Data System and Analysis Technology [J]. Journal of Software, 2014, 25(09):1889–1908.
Search in Google Scholar Back to article
Zhou Yanjun, Wang Shuangcheng, Wang Hui. Research on classifier based on Bayesian network [J]. Journal of Northeast Normal University (Natural Science Edition), 2003(02):21–27.
Search in Google Scholar Back to article
Xuanxuan Lin. Research on Enterprise Bankruptcy Prediction Method Based on XGBOOST Model [A]. Wuhan Zhicheng Times Cultural Development Co., Ltd. proceedings of 4th international conference on e-education, e-business and information management (EEIM 2021) [c]. Wuhan Zhicheng Times Cultural Development Co., Ltd.: Wuhan Zhicheng Times Cultural Development Co., Ltd., 2021:8.
Search in Google Scholar Back to article
Shenglong Li, Xiaojing Zhang. Research on orthopedic auxiliary classification and prediction model based on XGBoost algorithm [J]. Neural Computing and Applications, 2020, 32(prepublish):
Search in Google Scholar Back to article
Feng Chen, Chen Zhide. Application of xgboost and LSTM weighted combination model in sales forecast [J]. Computer system application, 2019, 28 (10): 226–232. Doi: 10.15888/j.cnki.csa.007091
Open DOI Search in Google Scholar Back to article
Shenglong Li, Xiaojing Zhang. Research on orthopedic auxiliary classification and prediction model based on XGBoost algorithm [J]. Neural Computing and Applications, 2020, 32(prepublish):
Search in Google Scholar Back to article
Wei Dong, Yimiao Huang, Barry Lehane, Guowei Ma. XGBoost algorithm-based prediction of concrete electrical resistivity for structural health monitoring [J]. Automation in Construction, 2020, 114(C):
Search in Google Scholar Back to article
Chixiang Wang, Junqi Guo. A data-driven framework for learners’ cognitive load detection using ECG-PPG physiological feature fusion and XGBoost classification [J]. Procedia Computer Science, 2019, 147:
Search in Google Scholar Back to article
Gao Yifan, Yu Wenzhe, Chao Pingfu, et al. Score prediction and recommendation based on comment analysis [J]. Journal of East China Normal University: Natural Science Edition, 2015 (3): 80–90. (Gao Yifan, Yu Wenzhe, Chao Pingfu, et al. Analyzing reviews for rating prediction and item recommendation [J]. Journal of East China Normal University: Natural Science, 2015 (3): 80–90.)
Search in Google Scholar Back to article
Li V., Costantino H., Rowland J., Yue L., Gupta S.. ML3 LASSO (Least Absolute Shrinkage and Selection Operator) and XGBoost (eXtreme Gradient Boosting) Models for Predicting Depression-Related Work Impairment in US Working Adults [J]. Value in Health, 2021, 24(S1).
Search in Google Scholar Back to article
Li V., Costantino H., Rowland J., Yue L., Gupta S.. ML3 LASSO (Least Absolute Shrinkage and Selection Operator) and XGBoost (eXtreme Gradient Boosting) Models for Predicting Depression-Related Work Impairment in US Working Adults[J]. Value in Health, 2021, 24(S1).
Search in Google Scholar Back to article
Deng Xiaoyi, Jin Chun, Han Qingping, et al. Collaborative filtering recommendation model based on situational clustering and user rating [J]. System engineering theory and practice, 2013, 33 (11):2945–2953. (Deng Xiaoyi, Jin Chun, Han Jim C, et al. Improved Collaborative Filtering Model Based on Context Clustering and User Ranking [J]. Systems Engineering —Theory & Practice, 2013, 33(11): 2945–2953.)
Search in Google Scholar Back to article
Zhang Hongli, Liu Jiying, Yang Sinan, et al. Research on scoring prediction model based on Internet user comments [J]. Data analysis and knowledge discovery, 2017, 1 (8): 48–58
Search in Google Scholar Back to article
McLachlan P, Munzner T, Koutsofios E, et al. LiveRAC: interactive visual exploration of system management time-series data[C] //Proceeding of the 26th International Conference on Human Factors in Computing Systems. New York: ACM Press, 2008: 1483–1492.
Search in Google Scholar Back to article
Agryzkov T, Oliver J L, Tortosa L, et al. Analyzing the commercial activities of a street network by ranking their nodes: a case study in Murcia, Spain [J]. International Journal of Geographical Information Science, 2014, 28(3/4): 479–495.
Search in Google Scholar Back to article
Qiu X, Suganthan P N, Amaratunga G A J. Ensemble incremental learning random vector functional link network for short-term electric load forecasting [J]. Knowledge-Based Systems, 2018, 145(4):182–196.
Search in Google Scholar Back to article
Wang J, Lou C, Yu R, et al. Research on hot micro-blog forecast based on XGBOOST and random forest [M]. Knowledge Science, Engineering and Management, KSEM 2018, Lecture Notes in Computer Science, Springer.
Search in Google Scholar Back to article
Li C, Chen Z. Y, Liu J. B, et al. Power load forecasting based on the combined model of LSTM and XGBoost [C]//PRAI’ 19:Proceedings of the 2019 the International Conference on Pattern Recognition and Artificial Intelligence. Wenzhou, China: ACM, 2019: 46–51.
Search in Google Scholar Back to article
Gómez-Ríos A, Luengo J, Herrera F. A study on the noise label influence in boosting algorithms: Adaboost, GBM and XGBoost [C]//International Conference on Hybrid Artificial Intelligence Systems (HAIS), 2017:268–280.
Search in Google Scholar Back to article
Yue Yanchun, Huang Tingzhu. Error reciprocal variable weight combination prediction method [J]. Journal of University of Electronic Science and technology, 2007 (S1): 349–351.
Search in Google Scholar Back to article
Cao y B, Xu J, Liu T Y, et al. Adapting ranking SVM to document retrieval[C] //Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM Press, 2006: 186–193.
Search in Google Scholar Back to article
Xu P P, Mei H H, Ren L, et al. ViDX: visual diagnostics of assembly line performance in smart factories [J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(1):291–300.
Search in Google Scholar Back to article
Sun Guanglu, song Zhichao, Liu Jinlai, Zhu Suxia, he Yongjun. Feature selection method based on maximum information coefficient and approximate Markov blanket [J]. Journal of automation, 2017, 43 (05): 795–805. Doi: 10.16383/j.aas.2017.c150851.
Open DOI Search in Google Scholar Back to article
Zhang Li, Yuan Yuyu, Wang Zong. Fcbf feature selection algorithm based on maximum correlation information coefficient [J]. Journal of Beijing University of Posts and telecommunications, 2018, 41 (04): 86–90. Doi: 10.13190/j.jbupt.2017-229.
Open DOI Search in Google Scholar Back to article

Feature Sorting Algorithm Based on XGBoost and MIC Combination Model

Abstract

Paradigm

My account