# Observational Data

Variance of Least Squares Estimator In this section we calculate the conditional variance of the OLS estimator. For any r 1 random vector Z deÖne the r r covariance matrix var(Z) = E (Z E (Z)) (Z E (Z))0 = E ZZ0 (E (Z)) (E (Z))0 and for any pair (Z; X) deÖne the conditional covariance matrix var(Z j X) = E (Z E (Z j X)) (Z E (Z j X))0 j X : We deÖne V b def = var b j X as the conditional covariance matrix of the regression coe¢ cient estimates. We now derive its form. The conditional covariance matrix of the n 1 regression error e is the n n matrix var(e j X) = E ee0 j X def = D: The i th diagonal element of D is E e 2 i j X = E e 2 i j xi = 2 i while the ijth o§-diagonal element of D is E (eiej j X) = E (ei j xi) E (ej j xj ) = 0 where the Örst equality uses independence of the observations (Assumption 4.1) and the second is (4.2). Thus D is a diagonal matrix with i th diagonal element 2 i : D = diag 2 1 ; :::; 2 n = 0 BBB@ 2 1 0 0 0 2 2 0 . . . . . . . . . . . . 0 0 2 n 1 C CHAPTER 4. LEAST SQUARES REGRESSION 109 In the special case of the linear homoskedastic regression model (4.3), then E e 2 i j xi = 2 i = 2 and we have the simpliÖcation D = In 2 : In general, however, D need not necessarily take this simpliÖed form. For any n r matrix A = A(X), var(A0y j X) = var(A0e j X) = A0DA: (4.9) In particular, we can write b = A0y where A = X (X0X) 1 and thus V b = var(b j X) = A0DA = X0X 1 X0DX X0X 1 : It is useful to note that X0DX = Xn i=1 xix 0 i 2 i ; a weighted version of X0X. In the special case of the linear homoskedastic regression model, D = In 2 , so X0DX = X0X 2 ; and the variance matrix simpliÖes to V b = X0X 1 2 : Theorem 4.2 Variance of Least-Squares Estimator In the linear regression model (Assumption 4.2) and i.i.d. sampling (Assumption 4.1) V b = var b j X = X0X 1 X0DX X0X 1 (4.10) where D is deÖned in (4.8). In the homoskedastic linear regression model (Assumption 4.3) and i.i.d. sampling (Assumption 4.1) V b = 2