Revue d'économie et de statistique appliquée
Volume 22, Numéro 1, Pages 31-46
2025-06-03

Regularized Linear Regression: A Study In Different Dimensional Frameworks



Authors : Mohamed Bachir Aoun . Fatiha Mezoued .

Abstract

This study concentrates on the evaluation of various regression model selection methods with sample sizes exceeding the number of features in one hand, and the number of features Surpassing sample size case in other, using parameters from [1] for low dimensional settings and [2] for high dimensional setup , focusing on regularization techniques and true subset selection accuracy. The experiments used an orthogonal, low correlation and high correlation design matrix under various number of features, with noise vectors generated from a Gaussian distribution, and varied noise levels, and observation counts. The assessment covered Least Absolute Shrinkage and Selection Operator (LASSO), Ridge, Elastic-Net, Minimax Concave Penalty (MCP) and Smoothly Clipped Absolute Deviation (SCAD) as sparse regularization methods, adaptive methods and Step-Wise selection methods. Performance was evaluated based on the frequency of correct subset selections, with higher frequencies reflecting superior methods as mentioned in [3]. In low dimensional data, all methods generally performed well across all scenarios, while some regularization methods were sensitive to sample size and noise. For high dimensional generation, all methods reach the perfect rate of recovery except the Step-Wise selection that shows perturbations and limitation in this case. This study not only provides valuable insights for practitioners in statistical model selection but also calls for future research to explore the application of these methods in real- world phenomena in all fields of study, which matches the parameters settings mentioned in this paper.

Keywords

Statistical Model ; High Dimension ; Linear Regression ; Sparse Regularization Methods ; Model Selection

Les articles similaires

Cross Validation Across Regularized Linear Regression

Mohamed Bachir Aoun .  Fatiha Mezoued . 
pages 595-631.

Analysis Of The Relationship Between The Transition To Renewable Energies And Sustainable Development Using Simple Linear Regression: A Case Study Of Algeria

Zemri Bouazza Elamine .  Khetib Sidi Mohamed . 
pages 624-637.

Modeling Process Between The Validity Of Multiple Linear Regression Results And The Accuracy Of Prediction Using Artificial Intelligence Tools: A Study Of A Middle Category Of Consumers.

Boutaghane Mohammed Abderrezak .  Yekhlef Abdellah . 
pages 194-214.

How Institutional Quality Influences The Economic Development-environmental Quality Relationship In Oil-rich Countries: A Non-linear Panel Threshold Regression Study

Boudjana Reda Hamza . 
pages 86-95.

Asymptotic Distributions Of Linear And Non-linear Combinations Of Extreme Order Statistics

Abdelhakim Necir .  Brahimi Brahim . 
pages 03-15.

Formats de citation