1.5.5.md

Solution to Review Question

by Qiang Gao, updated at May 11, 2017

---

Chapter 1 Finite-Sample Properties of OLS

Section 5 Relation to Maximum Likelihood

...

Review Question 1.5.5 (Likelihood equations for classical regression model)

We used the two-step procedure to derive the ML estimate for the classical regression model. An alternative way to find the ML estimator is to solve for the first-order conditions that set

$$ \begin{align} \frac{ \partial \log L( \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \beta } } } & = \frac{1}{ \gamma } \mathbf{X}' ( \mathbf{y} - \mathbf{X} \boldsymbol{ \beta } ) = \mathbf{0}, \tag{1.5.13a} \\ \frac{ \partial \log L ( \boldsymbol{ \theta } ) }{ \partial } & = -\frac{n}{ 2 \gamma } + \frac{1}{ 2 \gamma^2 } ( \mathbf{y} - \mathbf{X} \boldsymbol{ \beta } )' ( \mathbf{y} - \mathbf{X} \boldsymbol{ \beta } ) = 0. \tag{1.5.13b} \end{align} $$

The first-order conditions for the log likelihood is called the likelihood equations. Verify that the ML estimator given in Proposition 1.5 solves the likelihood equations.

Solution

Proposition 1.5 states that the ML estimator is

$$ \begin{align} \hat{ \boldsymbol{ \beta } } & = \mathbf{b} = ( \mathbf{X}' \mathbf{X} )^{-1} \mathbf{X}' \mathbf{y}, \tag{1} \\ \hat{ \gamma } & = \frac{ \mathbf{e}' \mathbf{e} }{n}. \tag{2} \end{align} $$

Substituting $$ \boldsymbol{ \beta } $$ and $$ \gamma $$ in (1.5.13a) with (1) and (2), we have

$$ \frac{n}{ \mathbf{e}' \mathbf{e} } \mathbf{X}' \mathbf{e} = \mathbf{0}. \tag{3} $$

Equation (3) holds because $$ \mathbf{X}' \mathbf{e} = \mathbf{0} $$, as stated in (1.2.3').

Substituting $$ \boldsymbol{ \beta } $$ and $$ \gamma $$ in (1.5.13b) with (1) and (2), we have

$$ -\frac{ n^2 }{ 2 \cdot \mathbf{e}' \mathbf{e} } + \frac{ n^2 }{2 \cdot \mathbf{e}' \mathbf{e} \cdot \mathbf{e}' \mathbf{e}} \mathbf{e}' \mathbf{e} = 0. \tag{4} $$

Equation (4) holds by cancelling terms.

---

Copyright ©2017 by Qiang Gao