Comparison of the Effects of Cross-validation Methods on Determining Performances of Classifiers Used in Diagnosing Congestive Heart Failure

Isler, Yalcin; Narin, Ali; Ozer, Mahmut

doi:10.1515/msr-2015-0027

Abstract

Congestive heart failure (CHF) occurs when the heart is unable to provide sufficient pump action to maintain blood flow to meet the needs of the body. Early diagnosis is important since the mortality rate of the patients with CHF is very high. There are different validation methods to measure performances of classifier algorithms designed for this purpose. In this study, k-fold and leave-one-out cross-validation methods were tested for performance measures of five distinct classifiers in the diagnosis of the patients with CHF. Each algorithm was run 100 times and the average and the standard deviation of classifier performances were recorded. As a result, it was observed that average performance was enhanced and the variability of performances was decreased when the number of data sections used in the cross-validation method was increased.

References

[1] U.S. National Library of Medicine. Heart failure (Medical Encyclopedia). http://www.nlm.nih.gov.
Search in Google Scholar
[2] Flavell, C., Stevenson, L.W. (2001). Take Heart with Heart Failure. Circulation, 104, 89.10.1161/hc4301.09913611684641
Search in Google Scholar
[3] Wilbur, J., James, P. (2005). Diagnosis and management of heart failure in the outpatient setting. Primary Care, 32, 1115-1129.10.1016/j.pop.2005.09.00516326230
Search in Google Scholar
[4] American Heart Association (2006). Heart Disease and Stroke Statistics-2006 Update: A Report From the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation, 113, 85-151.
Search in Google Scholar
[5] Isler, Y., Kuntalp, M. (2007). Combining Classical HRV Indices with Wavelet Entropy Measures Improves to Performance in Diagnosing Congestive Heart Failure. Computers in Biology and Medicine, 37(10), 1502-1510.10.1016/j.compbiomed.2007.01.01217359959
Search in Google Scholar
[6] Isler, Y., Kuntalp, M. (2010). Heart Rate Normalization in the Analysis of Heart Rate Variability in Congestive Heart Failure. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 224(3), 453-463.10.1243/09544119JEIM64220408490
Search in Google Scholar
[7] Yu, S.N., Lee, M.Y. (2012). Bispectral analysis and genetic algorithm for congestive heart failure recognition based on heart rate variability. Computers in Biology and Medicine, 42, 816-825.10.1016/j.compbiomed.2012.06.00522809682
Search in Google Scholar
[8] Yu, S.N., Lee, M.Y. (2012). Conditional mutual information-based feature selection for congestive heart failure recognition using heart rate variability. Computer Methods and Programs in Biomedicine, 108, 299-309.10.1016/j.cmpb.2011.12.01522261219
Search in Google Scholar
[9] Jovic, A., Bogunovic, N. (2011). Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features. Artificial Intelligence in Medicine, 51, 175-186.10.1016/j.artmed.2010.09.00520980134
Search in Google Scholar
[10] Pecchia, L., Melillo, P., Sansone, M., Bracale, M. (2011). Discrimination power of short-term heart rate variability measures for CHF assessment. IEEE Transactions on Information Technology in Biomedicine, 15(1), 40-46.10.1109/TITB.2010.209164721075731
Search in Google Scholar
[11] Narin, A., Isler, Y., Ozer, M. (2014). Investigating the performance improvement of HRV Indices in CHF using feature selection methods based on backward elimination and statistical significance. Computers in Biology and Medicine, 45, 72-79.10.1016/j.compbiomed.2013.11.01624480166
Search in Google Scholar
[12] Narin, A., Isler, Y. (2012). Effect of Principal Component Analysis on Diagnosing Congestive Heart Failure Patients using Heart Rate Records. In IEEE 20th Signal Processing and Communications Applications Conference (SIU2012), 18-20 April 2012, Fethiye / Mugla.10.1109/SIU.2012.6204735
Search in Google Scholar
[13] Isler, Y., Selver, M.A., Kuntalp, M. (2005). Effects of Detrending in Heart Rate Variability Analysis. In II. Muhendislik Bilimleri Genc Arastirmacilar Kongresi MBGAK’2005, 17-19 October 2005, Istanbul, 213-219.
Search in Google Scholar
[14] Duda, R.O., Hart, P.E., Stork, D.G. (2000). Pattern Classification, 2nd edition, New York. Wiley.
Search in Google Scholar
[15] Goldberger, A.L., et.al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals. Circulation, 101(23), e215-e220.10.1161/01.CIR.101.23.e215
Search in Google Scholar
[16] Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology. (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Circulation, 93, 1043-1065.
Search in Google Scholar
[17] Lomb, N.R. (1976). Least-squares frequency analysis of unequally spaced data. Astrophysics and Space Science, 39, 447-462.10.1007/BF00648343
Search in Google Scholar
[18] Quiraga, R.Q., Rosso, O.A., Basar, E., Schurmann, M. (2001). Wavelet entropy in event-related potentials: A new method shows ordering of EEG oscillations. Biological Cybernetics, 84(4), 291-299.10.1007/s004220000212
Search in Google Scholar
[19] Woo, M.A., Stevenson, W.G., Moser, D.K., Trelease, R.B., Harper, R.H. (1992). Patterns of beat-to-beat heart rate variability in advanced heart failure. American Heart Journal, 123, 704-710.10.1016/0002-8703(92)90510-3
Search in Google Scholar
[20] Kamen, P.W., Krum, H., Tonkin, A.M. (1996). Poincare plot of heart rate variability allows quantitative display of parasympathetic nervous activity. Clinical Science, 92, 201-208.10.1042/cs09102018795444
Search in Google Scholar
[21] Kamen, P.W. (1996). Heart rate variability. Australian Family Physician, 25, 1087-1094.
Search in Google Scholar
[22] Brennan, M., Palaniswami, M., Kamen, P. (2001). Do existing measures of poincare plot geometry reflect nonlinear features of heart rate variability? IEEE Transactions on Biomedical Engineering, 48(11), 1342-1347.10.1109/10.95933011686633
Search in Google Scholar
[23] Huikuri, H.V., Makikallio, T.H., Peng, C.K., Goldberger, A.L., Hintze, U., Moller, M. (2000). Fractal correlation properties of R-R interval dynamics and mortality in patients with depressed left ventricular function after an acute myocardial infarction. Circulation, 101, 47-53.10.1161/01.CIR.101.1.4710618303
Search in Google Scholar
[24] Acharya, U.R., Kannathal, N., Seng, O.W., Ping, L.Y., Chua, T. (2004). Heart rate analysis in normal subjects of various age groups. BioMedical Engineering OnLine, 3(24).10.1186/1475-925X-3-2449327815260880
Search in Google Scholar
[25] Caminal, P., Vallverdu, M., Giraldo, B., Benito, S., Vazquez, G., Voss, A. (2005). Optimized symbolic dynamics approach for the analysis of the respiratory pattern. IEEE Transactions on Biomedical Engineering, 52(11), 1832-1839.10.1109/TBME.2005.85629316285386
Search in Google Scholar
[26] Xu, J.H., Liu, Z.R., Liu, R. (1994). The measures of sequence complexity for EEG studies. Chaos, 4(11), 2111-2119.
Search in Google Scholar
[27] Richman, J.S., Randall, M.J. (2000). Physiological time-series analysis using approximate entropy and sample entropy. American Journal of Physiology - Heart and Circulatory Physiology, 278, H2039-H2049.10.1152/ajpheart.2000.278.6.H203910843903
Search in Google Scholar
[28] Akgul, A. (2003). Tibbi Arastirmalarda Istatistiksel Analiz Teknikleri: SPSS Uygulamalari (Statistical Analysis Techniques in Medical Researches: SPSS Experiments). 2nd edition, Ankara, Turkey, Seckin Yayincilik.
Search in Google Scholar
[29] Kohavi, R. (1995). A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In 14th International Joint Conference on Artificial Intelligence (IJCAI), 20-25 August 1995, Montreal, Quebec, Canada.
Search in Google Scholar

Comparison of the Effects of Cross-validation Methods on Determining Performances of Classifiers Used in Diagnosing Congestive Heart Failure

Abstract

Paradigm

My account