Have a personal or library account? Click to login
Difficulty Factors and Preprocessing in Imbalanced Data Sets: An Experimental Study on Artificial Data Cover

Difficulty Factors and Preprocessing in Imbalanced Data Sets: An Experimental Study on Artificial Data

Open Access
|Jun 2017

Abstract

In this paper we describe results of an experimental study where we checked the impact of various difficulty factors in imbalanced data sets on the performance of selected classifiers applied alone or combined with several preprocessing methods. In the study we used artificial data sets in order to systematically check factors such as dimensionality, class imbalance ratio or distribution of specific types of examples (safe, borderline, rare and outliers) in the minority class. The results revealed that the latter factor was the most critical one and it exacerbated other factors (in particular class imbalance). The best classification performance was demonstrated by non-symbolic classifiers, particular by k-NN classifiers (with 1 or 3 neighbors - 1NN and 3NN, respectively) and by SVM. Moreover, they benefited from different preprocessing methods - SVM and 1NN worked best with undersampling, while oversampling was more beneficial for 3NN.

DOI: https://doi.org/10.1515/fcds-2017-0007 | Journal eISSN: 2300-3405 | Journal ISSN: 0867-6356
Language: English
Page range: 149 - 176
Submitted on: Oct 19, 2016
Accepted on: Apr 24, 2017
Published on: Jun 16, 2017
Published by: Poznan University of Technology
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year

© 2017 Szymon Wojciechowski, Szymon Wilk, published by Poznan University of Technology
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.