# Orthogonal and Orthonormal Matrices

Bootstrap for 2SLS The standard bootstrap algorithm for IV, 2SLS and GMM generates bootstrap samples by sampling the triplets (y i ; x i ; z i ) independently and with replacement from the original sample f(yi ; xi ; zi) : i = 1; :::; ng. Sampling n such observations and stacking into observation matrices (y ; X ; Z ), the bootstrap 2SLS estimator is b 2sls = X0Z Z 0Z 1 Z 0X 1 X0Z Z 0Z 1 Z 0y : This is repeated B times to create a sample of B bootstrap draws. Given these draws, bootstrap statistics can be calculated. This includes the bootstrap estimate of variance, standard errors, and conÖdence intervals, including percentile, BC percentile, BCa and percentile-t. We now show that the bootstrap estimator has the same asymptotic distribution as the sample estimator. For overidentiÖed cases this demonstration requires a bit of extra care. This was Örst shown by Hahn (1996). The sample observations satisfy the model yi = x 0 i + ei E (ziei) = 0: The true value of in the population can be written as = E xiz 0 i E ziz 0 i 1 E zix 0 i 1 E xiz 0 i E ziz 0 i 1 E (ziyi): The true value in the bootstrap universe is obtained by replacing the population moments by the sample moments, which equals the 2SLS estimator E x i z 0 i E z i z 0 i 1 E z i x 0 i 1 E x i z 0 i E z i z 0 i 1 E (z i y i ) = 1 n X0Z 1 n Z 0Z 1 1 n Z 0X !1 1 n X0Z 1 n Z 0Z 1 1 n Z 0y = b 2sls: The bootstrap observations thus satisfy the equation y i = x 0 i b 2sls + e i : In matrix notation y = X0b 2sls + e : (12.45) Given a bootstrap triple (y i ; x i ; z i ) = (yj ; xj ; zj ) for some observation j, the true bootstrap error is e i = yj x 0 jb 2sls = ebj : It follows that E (z i e i ) = n 1Z 0be: (12.46) This is generally not equal to zero in the over-identiÖed case. This an an important complication. In over-identiÖed models the true observations satisfy the population condition E (ziei) = 0 but in the bootstrap sample E (z i e i CHAPTER 12. INSTRUMENTAL VARIABLES 433 apply the central limit theorem to the bootstrap estimator we will Örst have to recenter the moment condition. That is, (12.46) and the bootstrap CLT imply 1 p n Z 0e Z 0be = 1 p n Xn i=1 (z i e i E (z i e i )) d ! N (0; ) (12.47) where = E ziz 0 i e 2 i : Using (12.45) we can normalize the bootstrap estimator as p n b 2sls b 2sls = p n X0Z Z 0Z 1 Z 0X 1 X0Z Z 0Z 1 Z 0e = 1 n X0Z 1 n Z 0Z 1 1 n Z 0X !1 1 n X0Z 1 n Z 0Z 1 1 p n Z 0e Z 0be (12.48) + 1 n X0Z 1 n Z 0Z 1 1 n Z 0X !1 1 n X0Z 1 n Z 0Z 1 1 p n Z 0be : (12.49) Using the bootstrap WLLN, 1 n X0Z = 1 n X0Z + op(1) 1 n Z 0Z = 1 n Z 0Z + op(1): This implies (12.49) is equal to p n X0Z Z 0Z 1 Z 0X 1 X0Z Z 0Z 1 Z 0be + op(1) = 0 + op(1): The equality holds because the 2SLS Örst-order condition implies X0Z (Z 0Z) 1 Z 0be = 0. Also, combined with (12.47) we see that (12.48) converges in bootstrap distribution to QxzQ1 zz Qzx1 QxzQ1 zzN (0; ) = N (0;V ) where V is the 2SLS asymptotic variance from Theorem 12.2. This is the asymptotic distribution of p n