文章基本信息

标题：A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
本地全文：下载
作者：Joon Kwon ; Guillaume Lecué ; Matthieu Lerasle 等
期刊名称：Electronic Journal of Statistics
印刷版ISSN：1935-7524
出版年度：2021
卷号：15
期号：1
页码：1202-1227
DOI：10.1214/21-EJS1814
语种：English
出版社：Institute of Mathematical Statistics
摘要：Hyperparameter tuning and model selection are important steps in machine learning. Unfortunately, classical hyperparameter calibration and model selection procedures are sensitive to outliers and heavy-tailed data. In this work, we construct a selection procedure which can be seen as a robust alternative to cross-validation and is based on a median-of-means principle. Using this procedure, we also build an ensemble method which, trained with algorithms and corrupted heavy-tailed data, selects an algorithm, trains it with a large uncorrupted subsample and automatically tunes its hyperparameters. In particular, the approach can transform any procedure into a robust to outliers and to heavy-tailed data procedure while tuning automatically its hyperparameters.The construction relies on a divide-and-conquer methodology, making this method easily scalable even on a corrupted dataset. This method is tested with the LASSO which is known to be highly sensitive to outliers.
关键词：60K35; 62F35; heavy-tailed; robustness