# Regression Derivative

! X0X 1 : (4.34) These are often called the ìHC2îand ìHC3î estimators, as labeled. The four estimators HC0, HC1, HC2 and HC3 are collectively called robust, heteroskedasticityconsistent, or heteroskedasticity-robust covariance matrix estimators. The HC0 estimator was Örst developed by Eicker (1963) and introduced to econometrics by White (1980), and is sometimes called the Eicker-White or White covariance matrix estimator. The degree-of-freedom adjustment in HC1 was recommended by Hinkley (1977), and is the default robust covariance matrix estimator implemented in Stata. It is implement by the ì,rî option, for example by a regression executed with the command ìreg y x, rî. In applied econometric practice, this is the currently most popular covariance matrix estimator. The HC2 estimator was introduced by Horn, Horn and Duncan (1975) (and is implemented using the vce(hc2) option in Stata). The HC3 estimator was derived by MacKinnon and White (1985) from the jackknife principle (see Section 10.3), and by Andrews (1991a) based on the principle of leave-one-out cross-validation (and is implemented using the vce(hc3) option in Stata). Since (1 hii) 2 > (1 hii) 1 > 1 it is straightforward to show that Vb HC0 b < Vb HC2 b < Vb HC3 b (4.35) (See Exercise 4.10). The inequality A < B when applied to matrices means that the matrix B A is positive deÖnite. In general, the bias of the covariance matrix estimators is quite complicated, but they greatly simplify under the assumption of homoskedasticity (4.3). For example, using (4.22), E Vb HC0 b j X = X0X 1 Xn i=1 xix 0 iE eb 2 i j X ! X0X 1 = X0X 1 Xn i=1 xix 0 i (1 hii) 2 ! X0X 1 = X0X 1 2 X0X 1 Xn i=1 xix 0 ihii! CHAPTER 4. LEAST SQUARES REGRESSION 120 This calculation shows that Vb HC0 b is biased towards zero. By a similar calculation (again under homoskedasticity) we can calculate that the HC2 estimator is unbiased E Vb HC2 b j X = X0X 1 2 : (4.36) (See Exercise 4.11.) It might seem rather odd to compare the bias of heteroskedasticity-robust estimators under the assumption of homoskedasticity, but it does give us a baseline for comparison. Another interesting calculation shows that in general (that is, without assuming homoskedasticity) the HC3 estimator is biased away from zero. Indeed, using the deÖnition of the prediction errors (3.45) eei = yi x 0 ib (i) = ei x 0 i b (i) so ee 2 i = e 2 i 2x 0 i b (i) ei + x 0 i b (i) 2 : Note that ei and b (i) are functions of non-overlapping observations and are thus independent. Hence E b (i) ei j X = 0 and E ee 2 i j X = E e 2 i j X 2x 0 iE b (i) ei j X + E x 0 i b (i) 2 j X = 2 i + E x 0 i b (i) 2 j X 2 i : It follows that E Vb HC3 b j X = X0X 1 Xn i=1 xix 0 iE ee 2 i j X ! X0X 1 X0X 1 Xn i=1 xix 0 i 2 i ! X0X 1 = V b: This means that the HC3 estimator is conservative in the sense that it is weakly larger (in expectation) than the correct v