2026-04-17T11:32:47Zhttps://riubu.ubu.es/oai/requestoai:riubu.ubu.es:10259/48142022-04-29T12:02:45Zcom_10259_4219com_10259_5086com_10259_2604col_10259_4220
Kuncheva, Ludmila I. .
37a0866d-eced-4e47-913c-cdc9943f0a48
500
Rodríguez Diez, Juan José
477
500
2018-06-18T11:52:23Z
2018-09
2020-09
0031-3203
http://hdl.handle.net/10259/4814
10.1016/j.patcog.2018.03.012
High-dimensional data with very few instances are typical in many application domains. Selecting a highly discriminative subset of the original features is often the main interest of the end user. The widely-used feature selection protocol for such type of data consists of two steps. First, features are selected from the data (possibly through cross-validation), and, second, a cross-validation protocol is applied to test a classifier using the selected features. The selected feature set and the testing accuracy are then returned to the user. For the lack of a better option, the same low-sample-size dataset is used in both steps. Questioning the validity of this protocol, we carried out an experiment using 24 high-dimensional datasets, three feature selection methods and five classifier models. We found that the accuracy returned by the above protocol is heavily biased, and therefore propose an alternative protocol which avoids the contamination by including both steps in a single cross-validation loop. Statistical tests verify that the classification accuracy returned by the proper protocol is significantly closer to the true accuracy (estimated from an independent testing set) compared to that returned by the currently favoured protocol.
project RPG-2015-188 funded by The Leverhulme Trust, UK and by project TIN2015-67534-P (MINECO/FEDER, UE) funded by the Ministerio de Economía y Competitividad of the Spanish Government and European Union FEDER funds
application/pdf
eng
Elsevier
Pattern Recognition. 2018, V. 81, p. 660-673
https://doi.org/10.1016/j.patcog.2018.03.012
info:eu-repo/grantAgreement/MINECO/TIN2015-67534-P
info:eu-repo/grantAgreementThe Leverhulme Trust/RPG-2015-188
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
info:eu-repo/semantics/openAccess
Feature selection
Wide datasets
Experimental protoco
Training/testing
Cross-validation
Computer science
Informática
On feature selection protocols for very low-sample-size data
info:eu-repo/semantics/article
info:eu-repo/semantics/acceptedVersion
CC-LICENSE
license_rdf
license_rdf
application/rdf+xml; charset=utf-8
805
https://riubu.ubu.es/bitstream/10259/4814/8/license_rdf
4460e5956bc1d1639be9ae6146a50347
MD5
8
THUMBNAIL
Kuncheva-PR-2018.pdf.jpg
Kuncheva-PR-2018.pdf.jpg
IM Thumbnail
image/jpeg
4382
https://riubu.ubu.es/bitstream/10259/4814/7/Kuncheva-PR-2018.pdf.jpg
5b7cf7040ecc8f3cb6abaa5d0aedacaa
MD5
7
ORIGINAL
Kuncheva-PR-2018.pdf
Kuncheva-PR-2018.pdf
application/pdf
3615928
https://riubu.ubu.es/bitstream/10259/4814/1/Kuncheva-PR-2018.pdf
2860c55818a73043f7a7a689899c2750
MD5
1
LICENSE
license.txt
license.txt
text/plain; charset=utf-8
999
https://riubu.ubu.es/bitstream/10259/4814/5/license.txt
b295bcbce42e2caabeb0c623d3860c06
MD5
5
TEXT
Kuncheva-PR-2018.pdf.txt
Kuncheva-PR-2018.pdf.txt
Extracted text
text/plain
54343
https://riubu.ubu.es/bitstream/10259/4814/6/Kuncheva-PR-2018.pdf.txt
f9238edd647ccb722545c49cf2b71d5d
MD5
6
10259/4814
oai:riubu.ubu.es:10259/4814
2022-04-29 14:02:45.694
Repositorio Institucional de la Universidad de Burgos
bubrep@ubu.es
RWwgYXV0b3IgY29tbyDDum5pY28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgZGUgcHJvcGllZGFkIGludGVsZWN0dWFsIGRlIGxhIG9icmEsIG8gZGlzcG9uaWVuZG8gZGUgbG9zIGRlYmlkb3MgcGVybWlzb3MgZGUgbG9zIG90cm9zIHRpdHVsYXJlcywgc2kgbG9zIGh1YmllcmEsIHkgZW4gdmlydHVkIGRlIGxvcyBkZXJlY2hvcyBxdWUgbGUgY29uZmllcmUgbGEgbGVnaXNsYWNpw7NuIHZpZ2VudGUgc29icmUgcHJvcGllZGFkIGludGVsZWN0dWFsIHkgZGVyZWNob3MgZGUgYXV0b3IsIA0KQVVUT1JJWkEgYSBsYSBVbml2ZXJzaWRhZCBkZSBCdXJnb3MgYSBkaWZ1bmRpciwgZGUgbWFuZXJhIGdyYXR1aXRhLCBlbCBjb250ZW5pZG8gZGUgbG9zIGFyY2hpdm9zIGRpZ2l0YWxlcyBxdWUgY29ycmVzcG9uZGVuIGFsIGRvY3VtZW50byBkZXNjcml0byBhbnRlcmlvcm1lbnRlLCBjb24gY2Fyw6FjdGVyIG5vIGV4Y2x1c2l2byB5IGRlIG1hbmVyYSBww7pibGljYSBlbiBhY2Nlc28gYWJpZXJ0byBhIHRyYXbDqXMgZGUgSW50ZXJuZXQsIHBhcmEgbG8gcXVlIGxhIEJpYmxpb3RlY2EgcHJvY2VkZXLDoSBhIGFyY2hpdmFybG9zIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwuIEFzaW1pc21vIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgZGUgQnVyZ29zIGEgcmVhbGl6YXIgbGFzIHRyYW5zZm9ybWFjaW9uZXMgbmVjZXNhcmlhcyBkZSBmb3JtYXRvLCBubyBkZSBjb250ZW5pZG8sIHBhcmEgZ2FyYW50aXphciBsYSBwcmVzZXJ2YWNpw7NuIHkgZWwgYWNjZXNvIGVuIGVsIGZ1dHVyby4NCg0KRWwgYXV0b3IgZGlzcG9uZSwgZW4gdG9kbyBjYXNvLCBkZWwgZGVyZWNobyBhIHJldm9jYXIgZXN0YSBhdXRvcml6YWNpw7NuLg0KDQpMYSBjZXNpw7NuIGRlIGRlcmVjaG9zIGRlIGVzdGEgb2JyYSBzZSBlbmN1ZW50cmEgc3VqZXRhIGEgbGEgbGVnaXNsYWNpw7NuIHZpZ2VudGUgc29icmUgcHJvcGllZGFkIGludGVsZWN0dWFsIHkgZGVyZWNob3MgZGUgYXV0b3Iu