1.6.3.md

Solution to Review Question

by Qiang Gao, updated at May 15, 2017

---

Chapter 1 Finite-Sample Properties of OLS

Section 6 Generalized Least Squares (GLS)

...

Review Question 1.6.3

Derive the expression for $$ \mathrm{Var} ( \mathbf{b} \mid \mathbf{X} ) $$ for the generalized regression model. What is the relation of it to $$ \mathrm{Var} ( \hat{ \boldsymbol{ \beta } }_{ \mathrm{GLS} } \mid \mathbf{X} ) $$? Verify that Proposition 1.7(c) (efficiency of GLS) implies

$$ ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \mathbf{V} \mathbf{X} ( \mathbf{X}' \mathbf{X} )^{-1} \ge ( \mathbf{X}' \mathbf{V}^{-1} \mathbf{X} )^{-1}. $$

Solution

(a) For the generalized regression model,

$$ \begin{align} \mathrm{Var} ( \mathbf{b} \mid \mathbf{X} ) & = \mathrm{Var} ( \mathbf{b} - \boldsymbol{ \beta } \mid \mathbf{X} ) \ & = \mathrm{Var} ( ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \boldsymbol{ \varepsilon } \mid \mathbf{X} ) \ & = ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \mathrm{Var} ( \boldsymbol{ \varepsilon } \mid \mathbf{X} ) \mathbf{X} ( \mathbf{X}' \mathbf{X} )^{-1} \ & = \sigma^2 \cdot ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \mathbf{V} \mathbf{X} ( \mathbf{X}' \mathbf{X} )^{-1}. \tag{1} \end{align} $$

(b) According to the text,

$$ \mathrm{Var} ( \hat{ \boldsymbol{ \beta } }_{\mathrm{GLS}} \mid \mathbf{X} ) = \sigma^2 \cdot ( \mathbf{X}' \mathbf{V}^{-1} \mathbf{X} )^{-1}. \tag{1.6.6} $$

Because $$ \mathbf{b} $$ is unbiased, following the Gauss-Markov theorem in Proposition 1.7(c),

$$ \mathrm{Var} ( \mathbf{b} \mid \mathbf{X} ) \ge \mathrm{Var} ( \hat{ \boldsymbol{ \beta } }_{\mathrm{GLS}} \mid \mathbf{X} ) \tag{2} $$

in the matrix sense.

(c) Substituting (1) and (1.6.6) into (2), it is easy to derive that

$$ ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \mathbf{V} \mathbf{X} ( \mathbf{X}' \mathbf{X} )^{-1} \ge ( \mathbf{X}' \mathbf{V}^{-1} \mathbf{X} )^{-1}. $$

---

Copyright ©2017 by Qiang Gao