L1-L2 Regularization of Collinear Data

Abstract

Multiple linear regression analysis may be used to describe the relation of a variable (response variable) based on the score of several other variables (independent variables). The least squares estimate of the regression coefficients are unsteady in that replicate samples can give widely differing values of the regression coefficients if the predictor variables are highly correlated. Ridge and Lasso regression analysis are regularization techniques for eliminating the effect of high covariance from the regression analysis. They produce estimates that are biased but have smaller mean square errors between the coefficients and their estimates. The lasso and ridge trace plot of the coefficients versus _ and cross validation are some ways that helps to determine the value of regularization constant _ and regression coefficients based on the data. Ridge regression and Lasso regression help the analysis to a more trustable interpretation of the results of multiple regression with highly correlated covariates.