1.1.md

Solution to Analytical Exercise

by Qiang Gao, updated at May 17, 2017

---

Chapter 1 Finite-Sample Properties of OLS

...

Analytical Exercise 1.1 (Proof that $$ \mathbf{b} $$ minimizes $$ SSR $$)

Let $$ \mathbf{b} $$ be the OLS estimator of $$ \boldsymbol{ \beta } $$. Prove that, for any hypothetical estimator $$ \tilde{ \boldsymbol{ \beta } } $$ of $$ \boldsymbol{ \beta } $$,

$$ ( \mathbf{y} - \mathbf{X} \tilde{ \boldsymbol{ \beta } } )' ( \mathbf{y} - \mathbf{X} \tilde{ \boldsymbol{ \beta } } ) \ge ( \mathbf{y} - \mathbf{X} \mathbf{b} )' ( \mathbf{y} - \mathbf{X} \mathbf{b} ). $$

Solution

$$ \begin{align} & ( \mathbf{y} - \mathbf{X} \tilde{ \boldsymbol{ \beta } } )' ( \mathbf{y} - \mathbf{X} \tilde{ \boldsymbol{ \beta } } ) \ = & [ ( \mathbf{y} - \mathbf{X} \mathbf{b} ) + \mathbf{X} ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } ) ]' [ ( \mathbf{y} - \mathbf{X} \mathbf{b} ) + \mathbf{X} ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } ) ] \ = & ( \mathbf{y} - \mathbf{X} \mathbf{b} )' ( \mathbf{y} - \mathbf{X} \mathbf{b} ) + ( \mathbf{y} - \mathbf{X} \mathbf{b} )' \mathbf{X} ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } ) + ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } )' \mathbf{X}' ( \mathbf{y} - \mathbf{X} \mathbf{b} ) \ & + ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } )' \mathbf{X}' \mathbf{X} ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } ) \ = & ( \mathbf{y} - \mathbf{X} \mathbf{b} )' ( \mathbf{y} - \mathbf{X} \mathbf{b} ) + ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } )' \mathbf{X}' \mathbf{X} ( \mathbf{b} - \tilde{ \boldsymbol{ \beta } } ) \qquad \text{ ( because $ \mathbf{X}' \mathbf{y} = \mathbf{X}' \mathbf{X} \mathbf{b} $ ) } \ \ge & ( \mathbf{y} - \mathbf{X} \mathbf{b} )' ( \mathbf{y} - \mathbf{X} \mathbf{b} ). \qquad \text{ (because $ \mathbf{X}' \mathbf{X} $ is positive definite and $ \tilde{ \boldsymbol{ \beta } } $ could $= \mathbf{b}$) } \end{align} $$

---

Copyright ©2017 by Qiang Gao