Solution to Review Question by Qiang Gao, updated at May 15, 2017
Chapter 1 Finite-Sample Properties of OLS Section 5 Relation to Maximum Likelihood ...
Review Question 1.5.1 (Use of regularity conditions) Assuming that taking expectations (i.e. taking integrals) and differentiation can be interchanged, prove that the expected value of the score vector,
$$
\mathbf{s} ( \tilde{ \boldsymbol{ \theta } } ) \equiv
\frac{ \partial \log L ( \tilde{ \boldsymbol{ \theta } } ) }
{ \partial \tilde{ \boldsymbol{ \theta } } },
\tag{1.5.9}
$$
if evaluated at the true parameter value $$ \boldsymbol{\theta} $$, is zero.
We start from the identity on pdf,
$$
\int f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) , d \mathbf{z} = 1.
\tag{1}
$$
Differentiate both sides of (1) with respect to $$ \tilde{ \boldsymbol{ \theta } } $$ and use the regularity condition, which allows us to interchange integration and differentiation, we obtain
$$
\int \frac{ \partial f ( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }
{ \partial \tilde{ \boldsymbol{ \theta } } }
, d \mathbf{z} = 0.
\tag{2}
$$
Dividing $$ f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) $$ and multiplying $$ f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) $$ on the integrand of (2),
$$
\int \frac{1}{ f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }
\frac{ \partial f ( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }
{ \partial \tilde{ \boldsymbol{ \theta } } } \cdot
f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } )
, d \mathbf{z} = 0.
\tag{3}
$$
From basic calculus, the score vector function (1.5.9) can be written as
$$
\mathbf{s} ( \tilde{ \boldsymbol{ \theta } } ) =
\frac{ \partial \log L ( \tilde{ \boldsymbol{ \theta } } ) }
{ \partial \tilde{ \boldsymbol{ \theta } } } =
\frac{ \partial \log f ( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }{ \partial \tilde{ \boldsymbol{ \theta } } } =
\frac{1}{ f( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }
\frac{ \partial f ( \mathbf{z}; \tilde{ \boldsymbol{ \theta } } ) }
{ \partial \tilde{ \boldsymbol{ \theta } } }.
\tag{4}
$$
Combining (3) and (4), evaluating at the true parameter value $$\boldsymbol{ \theta }$$ , using the definition of expectation,
$$
\int \frac{1}{ f( \mathbf{z}; \boldsymbol{ \theta } ) }
\frac{ \partial f ( \mathbf{z}; \boldsymbol{ \theta } ) }
{ \partial \boldsymbol{ \theta } } \cdot
f( \mathbf{z}; \boldsymbol{ \theta } )
, d \mathbf{z} =
\int \mathbf{s} ( \boldsymbol{ \theta } ) \cdot
f( \mathbf{z}; \boldsymbol{ \theta } )
, d \mathbf{z} =
\mathrm{E} [ \mathbf{s} ( \boldsymbol{ \theta } ) ]
= 0.
$$
Copyright ©2017 by Qiang Gao