# Conditional Expectation .

e. 4.10 Residuals What are some properties of the residuals ebi = yi x 0 ib and prediction errors eei = yi x 0 ib (i) , at least in the context of the linear regression model? Recall from (3.25) that we can write the residuals in vector notation as be = M e where M = In X (X0X) 1 X0 is the orthogonal projection matrix. Using the properties of conditional expectation E (be j X) = E (M e j X) = ME (e j X) = 0 and var (be j X) = var (M e j X) = M var (e j X)M = MDM (4.20) where D is deÖned in (4.8). We can simplify this expression under the assumption of conditional homoskedasticity E e 2 i j xi = 2 : In this case (4.20) simpliÖes to var (be j X) = M 2 : (4.21) In particular, for a single observation i; we can Önd the (conditional) variance of ebi by taking the i th diagonal element of (4.21). Since the i th diagonal element of M is 1 hii as deÖned in (3.41) we obtain var (ebi j X) = E eb 2 i j X = (1 hii) 2 : (4.22) As this variance is a function of hii and hence xi , the residuals ebi are heteroskedastic even if the errors ei are homoskedastic. Notice as well that (4.22) implies eb 2 i is a biased estimator of 2 . Similarly, recall from (3.46) that the prediction errors eei = (1 hii) 1 ebi can be written in vector notation as ee = Mbe where M is a diagonal matrix with i th diagonal element (1 hii) 1 : Thus ee = MM e: We can calculate that E (ee j X) = MME (e j X) = 0 and var (ee j X) = MM var (e j X)MM = MMDMM which simpliÖes under homoskedasticity to var (ee j X) = M CHAPTER 4. LEAST SQUARES REGRESSION 114 The variance of the i th prediction error is then var (eei j X) = E ee 2 i j X = (1 hii) 1 (1 hii) (1 hii) 1 2 = (1 hii) 1 2 : A residual with constant conditional variance can be obtained by rescaling. The standardized residuals are ei = (1 hii) 1=2 ebi ; (4.23) and in vector notation e = (e1; :::; en) 0 = M1=2M e: (4.24) From our above calculations, under homoskedasticity, var (e j X) = M1=2MM1=2 2 and var (ei j X) = E e 2 i j X = 2 and thus these standardized residuals have the same bias and variance as the original errors when the latter are ho