2024-03-29T11:34:33Zhttps://riubu.ubu.es/oai/requestoai:riubu.ubu.es:10259/57662022-11-21T12:30:19Zcom_10259_4219com_10259_5086com_10259_2604col_10259_4220
Experimental evaluation of ensemble classifiers for imbalance in Big Data
Juez Gil, Mario
Arnaiz González, Álvar
Rodríguez Diez, Juan José
García Osorio, César
Unbalance
Imbalance
Ensemble
Resampling
Big Data
Spark
Datasets are growing in size and complexity at a pace never seen before, forming ever larger datasets known as Big Data. A common problem for classification, especially in Big Data, is that the numerous examples of the different classes might not be balanced. Some decades ago, imbalanced classification was therefore introduced, to correct the tendency of classifiers that show bias in favor of the majority class and that ignore the minority one. To date, although the number of imbalanced classification methods have increased, they continue to focus on normal-sized datasets and not on the new reality of Big Data. In this paper, in-depth experimentation with ensemble classifiers is conducted in the context of imbalanced Big Data classification, using two popular ensemble families (Bagging and Boosting) and different resampling methods. All the experimentation was launched in Spark clusters, comparing ensemble performance and execution times with statistical test results, including the newest ones based on the Bayesian approach. One very interesting conclusion from the study was that simpler methods applied to unbalanced datasets in the context of Big Data provided better results than complex methods. The additional complexity of some of the sophisticated methods, which appear necessary to process and to reduce imbalance in normal-sized datasets were not effective for imbalanced Big Data.
2021-05-14T11:34:01Z
2021-05-14T11:34:01Z
2021-05-14T11:34:01Z
2021-09
info:eu-repo/semantics/article
1568-4946
http://hdl.handle.net/10259/5766
10.1016/j.asoc.2021.107447
eng
Applied Soft Computing. 2021, V. 108, 107447
https://doi.org/10.1016/j.asoc.2021.107447
info:eu-repo/grantAgreement/Fundación Bancaria Caixa d'Estalvis i Pensions de Barcelona//LCF%2FPR%2FPR18%2F51130007
info:eu-repo/grantAgreement/Junta de Castilla y León//BU055P20//Métodos y Aplicaciones Industriales del Aprendizaje Semisupervisado
http://creativecommons.org/licenses/by-nc-nd/4.0/
info:eu-repo/semantics/openAccess
Attribution-NonCommercial-NoDerivatives 4.0 Internacional
Elsevier