Solution to Review Question by Qiang Gao, updated at May 15, 2017
Chapter 1 Finite-Sample Properties of OLS Section 5 Relation to Maximum Likelihood ...
Review Question 1.5.4 (Information matrix equality for classical regression model) Verify the information matrix equality
$$
\mathbf{I} ( \boldsymbol{ \theta } ) =
\mathrm{E} \left[
\frac{ \partial^2 \log L( \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } , \partial \tilde{ \boldsymbol{ \theta } }' }
\right]
\tag{1.5.11}
$$
for the linear regression model.
Comment : The information matrix equality is an identity, it is not specific to the linear regression model.
We begin with the identity
$$
1 = \oint_{ \Omega } f( \mathbf{z} ; \boldsymbol{ \theta } ) , d \mathbf{z}.
$$
Taking derivative with respect to $$ \boldsymbol{ \theta } $$ and interchange differentiation and integration,
$$
0 = \oint_{\Omega} \frac{ \partial f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } } , d \mathbf{z} =
\oint_{\Omega} \frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } } f( \mathbf{z}; \boldsymbol{ \theta } ) , d \mathbf{z}.
$$
Taking derivative with respect to $$ \boldsymbol{ \theta }' $$ and interchange differentiation and integration,
$$
\begin{align}
0 & = \oint_{\Omega} \frac{ \partial^2 \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } , \partial \tilde{ \boldsymbol{ \theta } }' } f( \mathbf{z}; \boldsymbol{ \theta } ) , d \mathbf{z} +
\oint_{\Omega} \frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } }
\frac{ \partial f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } }' } , d \mathbf{z}
\ & =
\oint_{\Omega} \frac{ \partial^2 \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } , \partial \tilde{ \boldsymbol{ \theta } }' } f( \mathbf{z}; \boldsymbol{ \theta } ) , d \mathbf{z} +
\oint_{\Omega} \frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } }
\frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } }' } f( \mathbf{z}; \boldsymbol{ \theta } ) , d \mathbf{z}.
\tag{1}
\end{align}
$$
Because
$$
\begin{align}
\mathbf{I} ( \boldsymbol{ \theta } ) & \equiv
\mathrm{E} [ \mathbf{s} ( \boldsymbol{ \theta } ) \mathbf{s} ( \boldsymbol{ \theta } )' ]
\tag{1.5.10}
\ & =
\oint_{\Omega} \frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } } }
\frac{ \partial \log f( \mathbf{z}; \boldsymbol{ \theta } ) }{ \partial \tilde{ \boldsymbol{ \theta } }' } f( \mathbf{z}; \boldsymbol{ \theta } ) , d \mathbf{z},
\end{align}
$$
and $$ L( \boldsymbol{ \theta } ) \equiv f( \mathbf{z}; \boldsymbol{ \theta } ) $$, substituting into (1) and rearranging terms, we get (1.5.11).
Copyright ©2017 by Qiang Gao