Ridge Regression Biased Estimation for Nonorthogonal Problems 岭回归：非正交问题的偏估计.pdf

TECHNOMETIIICS VOL. 12, No. 1 FEBRUARY1970 Ridge Regression: Biased Estimation for Nonorthogonal Problems ARTHUR E. H~ERL AND ROBERT W. KENNARD University of Delaware and E. 1. du Pont de Nemours Co. In multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory, if not incor- rect, if the prediction vectors are not orthogonal. Proposed is an estimation procedure based on adding small positive quantities to the diagonal of X’X. Introduced is the ridge trace, a method for showing in two dimensions the effects of nonorthogonality. It is then shown how to augment X’X to obtain biased estimates with smaller mean square error. 0. INTRODUCTION Consider the standard model for multiple linear regression, Y = XQ + e, where it is assumed that X is (n X p) and of rank p, Q is (p X 1) and unknown, E[E] = 0, and E[ze’] = ~~1%. If an observation on the factors is denoted by xv = {T”, X2”, * * *, z,,), the general form X@is ( cpE1 @,0,(x,)) where the Bi are functions free of unknown parameters. The usual estimation procedure for the unknown @is Gauss-Markov-linear functions of Y = {LJ~] that are unbiased and have minimum variance. This estimation procedure is a good one if X’X, when in the form of a correlation matrix, is nearly a unit matrix. However, if X’X is not nearly a unit matrix,