RT info:eu-repo/semantics/article
T1 Approx-SMOTE: Fast SMOTE for Big Data on Apache Spark
A1 Juez Gil, Mario
A1 Arnaiz González, Álvar
A1 Rodríguez Diez, Juan José
A1 López Nozal, Carlos
A1 García Osorio, César
K1 SMOTE
K1 Imbalance
K1 Spark
K1 Big data
K1 Data mining
K1 Informática
K1 Computer science
AB One of the main goals of Big Data research, is to find new data mining methods that are able to process large amounts of data in acceptable times. In Big Data classification, as in traditional classification, class imbalance is a common problem that must be addressed, in the case of Big Data also looking for a solution that can be applied in an acceptable execution time. In this paper we present Approx-SMOTE, a parallel implementation of the SMOTE algorithm for the Apache Spark framework. The key difference with the original SMOTE, besides parallelism, is that it uses an approximated version of k-Nearest Neighbor which makes it highly scalable. Although an implementation of SMOTE for Big Data already exists (SMOTE-BD), it uses an exact Nearest Neighbor search, which does not make it entirely scalable. Approx-SMOTE on the other hand is able to achieve up to 30 times faster run times without sacrificing the improved classification performance offered by the original SMOTE.
PB Elsevier
SN 0925-2312
YR 2021
FD 2021-11
LK http://hdl.handle.net/10259/6206
UL http://hdl.handle.net/10259/6206
LA eng
NO “La Caixa” Foundation, under agreement LCF/PR/PR18/51130007. This work was supported by the Junta de Castilla y León under project BU055P20 and by the Ministry of Science and Innovation of Spain under project PID2020-119894 GB-I00, co-financed through European Union FEDER funds. It also was supported through Consejería de Educación of the Junta de Castilla y León and the European Social Fund through a pre-doctoral grant (EDU/1100/2017). This material is based upon work supported by Google Cloud.
DS Repositorio Institucional de la Universidad de Burgos
RD 23-abr-2024