Skip to main content
Have a personal or library account? Click to login
Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms Cover

Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms

International Journal of Applied Mathematics and Computer Science
Volume 22 (2012): Issue 4 (December 2012)
By: ,  ,   and    
Open Access
|Dec 2012
References

References

  1. Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L. and Herrera, F. (2011). Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework,(2-3): 255-287.
    Search in Google Scholar
  2. Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M., Ventura, S., Garrell, J., Otero, J., Romero, C., Bacardit, J., Rivas, V., Fernández, J. and Herrera, F. (2009). KEEL: A software tool to assess evolutionary algorithms to data mining problems,(3): 307-318.
    Open DOISearch in Google Scholar
  3. Anderson, T. and Darling, D. (1954). A test of goodness-of-fit,(268): 765-769.
    Open DOISearch in Google Scholar
  4. Anscombe, F. and Glynn, W. (1983). Distribution of the kurtosis statistic b2 for normal samples,(1): 227-234.
    Open DOISearch in Google Scholar
  5. Baruque, B., Porras, S. and Corchado, E. (2011). Hybrid classification ensemble using topology-preserving clustering,(3): 329-344.
    Open DOISearch in Google Scholar
  6. Bergmann, G. and Hommel, G. (1988). Improvements of general multiple test procedures for redundant systems of hypotheses,P. Bauer, G. Hommel and E. Sonnemann (Eds.),, Springer-Verlag, Berlin, pp. 100-115.
    Open DOISearch in Google Scholar
  7. Broomhead, D. and Lowe, D. (1998). Multivariable functional interpolation and adaptive networks,: 321-355.
    Search in Google Scholar
  8. Czarnowski, I. and Je˛drzejowicz, P. (2011). Application of agent-based simulated annealing and tabu search procedures to solving the data reduction problem,(1): 57-68, DOI: 10.2478/v10006-011-0004-3.
    Open DOISearch in Google Scholar
  9. D’Agostino, R. (1970). Transformation to normality of the null distribution of g1,(3): 679-681.
    Open DOISearch in Google Scholar
  10. D’Agostino, R., Belanger, A. and D’Agostino Jr., R. (1990). A suggestion for using powerful and informative tests of normality,(4): 316-321.
    Open DOISearch in Google Scholar
  11. Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets,: 1-30.
    Search in Google Scholar
  12. Derrac, J., García, S., Molina, D. and Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms,: 3-18.
    Open DOISearch in Google Scholar
  13. Dunn, O. (1961). Multiple comparisons among means,(238): 52-64.
    Open DOISearch in Google Scholar
  14. Finner, H. (1993). On a monotonicity problem in step-down multiple test procedures,(423): 920-923.
    Open DOISearch in Google Scholar
  15. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance,(200): 675-701.
    Open DOISearch in Google Scholar
  16. García, S., Fernández, A., Luengo, J. and Herrera, F. (2009). A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability,(13): 959-977.
    Open DOISearch in Google Scholar
  17. García, S., Fernández, A. and Luengo, J.and Herrera, F. (2010). Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power,: 2044-2064.
    Open DOISearch in Google Scholar
  18. García, S. and Herrera, F. (2008). An extension on “Statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons,: 2677-2694.
    Search in Google Scholar
  19. Graczyk, M., Lasota, T., Telec, Z. and Trawin´ski, B. (2010). Nonparametric statistical analysis of machine learning algorithms for regression problems,R. Setchi, I. Jordanov, R.J. Howlett and L.C. Jain (Eds.),, Lecture Notes in Artificial Intelligence, Vol. 6276, Springer, Heidelberg, pp. 111-120.
    Open DOISearch in Google Scholar
  20. Graczyk, M., Lasota, T. and Trawin´ski, B. (2009). Comparative analysis of premises valuation models using KEEL, RapidMiner, and WEKA,N.T. Nguyen, R. Kowalczyk and S.-M. Chen (Eds.),, Lecture Notes in Artificial Intelligence, Vol. 5796, Springer, Heidelberg, pp. 800-812.
    Open DOISearch in Google Scholar
  21. Hill, T. and Lewicki, P. (2007)., StatSoft, Tulsa.
    Search in Google Scholar
  22. Hochberg, Y. (1988). A Sharper Bonferroni procedure for multiple tests of significance,(4): 800-802.
    Open DOISearch in Google Scholar
  23. Hodges, J. and Lehmann, E. (1962). Ranks methods for combination of independent experiments in analysis of variance,: 482-497.
    Open DOISearch in Google Scholar
  24. Holland, B. and Copenhaver, M. (1987). An improved sequentially rejective Bonferroni test procedure,(2): 417-423.
    Open DOISearch in Google Scholar
  25. Holm, S. (1979). A simple sequentially rejective multiple test procedure,: 65-70.
    Search in Google Scholar
  26. Hommel, G. (1988). A stagewise rejective multiple test procedure based on a modified Bonferroni test,(2): 383-386.
    Open DOISearch in Google Scholar
  27. Hommel, G.and Bernhard, G. (1994). A rapid algorithm and a computer program for multiple test procedures using procedures using logical structures of hypotheses,: 213-216.
    Open DOISearch in Google Scholar
  28. Igel, C. and Hüsken, M. (2003). Empirical evaluation of the improved RPROP learning algorithm,: 105-123.
    Open DOISearch in Google Scholar
  29. Iman, R. and Davenport, J. (1980). Approximations of the critical region of the Friedman statistic,: 571-595.
    Open DOISearch in Google Scholar
  30. Jackowski, K. and Woz´niak, M. (2010). Method of classifier selection using the genetic approach,(2): 114-128.
    Open DOISearch in Google Scholar
  31. Jarque, C. and Bera, A. (1987). A test for normality of observations and regression residuals,(2): 163-172.
    Open DOISearch in Google Scholar
  32. Kajdanowicz, T. and Kazienko, P. (2011). Boosting-based sequential output prediction,(3): 293-307.
    Open DOISearch in Google Scholar
  33. Keskin, S. (2006). Comparison of several univariate normality tests regarding type I error rate and power of the test in simulation based small samples,(5): 296-300.
    Search in Google Scholar
  34. Król, D., Lasota, T., Trawin´ski, B. and Trawin´ski, K. (2008). Investigation of evolutionary optimization methods of TSK fuzzy model for real estate appraisal,(3): 111-128.
    Open DOISearch in Google Scholar
  35. Krzystanek, M., Lasota, T. and Trawin´ski, B. (2009). Comparative analysis of evolutionary fuzzy models for premises valuation using KEEL,N.T. Nguyen, R. Kowalczyk and S.-M. Chen (Eds.),, Lecture Notes in Artificial Intelligence, Vol. 5796, Springer, Heidelberg, pp. 838-849.
    Open DOISearch in Google Scholar
  36. Lasota, T., Mazurkiewicz, J., Trawin´ski, B. and Trawin´ski, K. (2010). Comparison of data driven models for the validation of residential premises using KEEL,(1): 3-16.
    Open DOISearch in Google Scholar
  37. Lasota, T., Telec, Z., Trawin´ski, B. and Trawin´ski, K. (2011). Investigation of the ets evolving fuzzy systems applied to real estate appraisal,(2-3): 229-253.
    Search in Google Scholar
  38. Li, J. (2008). A two-step rejection procedure for testing multiple hypotheses,(6): 1521-1527.
    Open DOISearch in Google Scholar
  39. Lilliefors, H. (1967). On the Kolmogorov-Smirnov test for normality with mean and variance unknown,(318): 399-402.
    Open DOISearch in Google Scholar
  40. Luengo, J., García, S. and Herrera, F. (2009). A study on the use of statistical tests for experimentation with neural networks: Analysis of parametric test conditions and non-parametric tests,: 7798-7808.
    Open DOISearch in Google Scholar
  41. Lughofer, E., Trawin´ski, B., Trawin´ski, K., Kempa, O. and Lasota, T. (2011). On employing fuzzy modeling algorithms for the valuation of residential premises,: 5123-5142.
    Open DOISearch in Google Scholar
  42. Moller, F. (1990). A scaled conjugate gradient algorithm for fast supervised learning,: 525-533.
    Open DOISearch in Google Scholar
  43. Motulsky, H. (2010)., 2nd Edn., Oxford University Press, New York, NY.
    Search in Google Scholar
  44. Nemenyi, P.B. (1963).Ph.D. thesis, Princeton University, Princeton, NJ.
    Search in Google Scholar
  45. Plackett, R. (1983). Karl Pearson and the chi-squared test,(1): 59-72.
    Open DOISearch in Google Scholar
  46. Plat, J. (1991). A resource allocating network for function interpolation,(2): 213-225.
    Open DOISearch in Google Scholar
  47. Quade, D. (1979). Using weighted rankings in the analysis of complete blocks with additive block effects,: 680-683.
    Open DOISearch in Google Scholar
  48. Romão, X., Delgado, R. and Costa, A. (2010). An empirical power comparison of univariate goodness-of-fit tests for normality,(5): 545-591.
    Open DOISearch in Google Scholar
  49. Rom, D. (1990). A sequentially rejective test procedure based on a modified Bonferroni inequality,(3): 663-665.
    Open DOISearch in Google Scholar
  50. Royston, P. (1993). A pocket-calculator algorithm for the Shapiro-Francia test for non-normality: An application to medicine,(2): 181-184.
    Open DOISearch in Google Scholar
  51. Salzberg, S. (1997). On comparing classifiers: Pitfalls to avoid and a recommended approach,: 317-327.
    Open DOISearch in Google Scholar
  52. Shaffer, J. (1986). Modified sequentially rejective multiple test procedures,(395): 826-831.
    Open DOISearch in Google Scholar
  53. Shapiro, S. and Wilk, M. (1965). An analysis of variance test for normality (complete samples),(3/4): 591-611.
    Open DOISearch in Google Scholar
  54. Sheskin, D. (2011)., 5th Edn., Chapman & Hall/CRC, Boca Raton, FL.
    Search in Google Scholar
  55. Smętek, M. and Trawin´ski, B. (2011). Investigation of genetic algorithms with self-adaptive crossover, mutation, and selection,E. Corchado, M. Kurzyn´ski and M. Woz´niak (Eds.),, Lecture Notes in Artificial Intelligence, Vol. 6678, Springer, Heidelberg, pp. 116-123.
    Open DOISearch in Google Scholar
  56. Smotroff, I., Friedman, D. and Connolly, D. (1991). Self organizing modular neural networks,, pp. 187-192.
    Search in Google Scholar
  57. Székely, G.J. and Rizzo, M. (2005). A new test for multivariate normality,(1): 58-80.
    Open DOISearch in Google Scholar
  58. Tanweeer-Ul-Islam (2011). Normality testing-A new direction,(3): 115-118.
    Search in Google Scholar
  59. Thodę H. (2002)., Marcel Dekker, New York, NY.
    Search in Google Scholar
  60. Troç, M. and Unold, O. (2010). Self-adaptation of parameters in a learning classifier system ensemble machine,(1): 157-174, DOI: 10.2478/v10006-010-0012-8.
    Open DOISearch in Google Scholar
  61. Wilcoxon, F. (1945). Individual comparisons by ranking methods,: 80-83.
    Open DOISearch in Google Scholar
  62. Wright, S. (1992). Adjusted p-values for simultaneous inference,: 1005-1013.
    Open DOISearch in Google Scholar
  63. Yazici, B. and Yolacan, S. (2007). A comparison of various tests of normality,(2): 175-183.
    Open DOISearch in Google Scholar
  64. Zaman, M. and Hirose, H. (2011). Classification performance of bagging and boosting type ensemble methods with small training sets,(3): 277-292.
    Open DOISearch in Google Scholar
  65. Zar, J. (2009)., 5th Edn., Prentice Hall, Upper Saddle River, NJ.
    Search in Google Scholar
DOI: https://doi.org/10.2478/v10006-012-0064-z | Journal eISSN: 2083-8492 | Journal ISSN: 1641-876X
Journal RSS Feed
Language: English
Page range: 867 - 881
Published on: Dec 28, 2012
Published by: University of Zielona Góra
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Keywords:
machine learning,
nonparametric statistical tests,
statistical regression,
neural networks,
multiple comparison tests.
Related subjects:
Mathematics,
Applied mathematics

© 2012 Bogdan Trawiński, Magdalena Smętek, Zbigniew Telec, Tadeusz Lasota, published by University of Zielona Góra
This work is licensed under the Creative Commons License.

Volume 22 (2012): Issue 4 (December 2012)
Previous article
Next article