Using a collection of simulated and real benchmarks, we compare
Bayesian and frequentist regularization approaches under a low informative con-
straint when the number of variables is almost equal to the number of observations
on simulated and real datasets. This comparison includes new global noninforma-
tive approaches for Bayesian variable selection built on Zellner's g-priors that are
similar to Liang et al. (2008). The interest of those calibration-free proposals is
discussed. The numerical experiments we present highlight the appeal of Bayesian
regularization methods, when compared with non-Bayesian alternatives. They
dominate frequentist methods in the sense that they provide smaller prediction
errors while selecting the most relevant variables in a parsimonious way.