2.1.2.md

Solution to Review Question

by Qiang Gao, updated at June 8, 2018

---

Chapter 2 Large-Sample Theory

Section 1 Review of Limit Theorems for Sequences of Random Variables

...

Review Question 2.1.2 (Alternative definition of convergence for vector sequences)

(a) Verify that the definition in the text of “$$ \mathbf{z}n \to{m.s.} \mathbf{z} $$” is equivalent to

$$ \lim_{n \to \infty} \operatorname{E} [ ( \mathbf{z}_n - \mathbf{z} )'( \mathbf{z}_n - \mathbf{z} ) ] = 0. $$

Hint: $$ \operatorname{E} [ ( \mathbf{z}n - \mathbf{z} )'( \mathbf{z}n - \mathbf{z} ) ] = \operatorname{E} [ ( z{n1} - z_1 )^2 ] + \cdots + \operatorname{E} [ ( z{nK} - z_K )^2 ] $$, where $$K$$ is the dimension of $$\mathbf{z}$$.

(b) Similarly, verify that the definition in the text of “$$ \mathbf{z}_n \to_p \boldsymbol{\alpha} $$” is equivalent to

$$ \lim_{n \to \infty} \operatorname{Prob} \left( ( \mathbf{z}_n - \boldsymbol{\alpha} )' ( \mathbf{z}_n - \boldsymbol{\alpha} ) > \varepsilon \right) = 0, $$

for any $$ \varepsilon > 0 $$.

Solution

(a) If $$ \mathbf{z}n \to{m.s.} \mathbf{z} $$, by definition in the text,

$$ \lim_{n \to \infty} \operatorname{E} [ ( z_{nk} - z_k )^2 ] = 0, \text{ for $k = 1, \ldots, K$}, $$

then we can show

$$ \begin{align} & \color{white}{=} \lim_{n \to \infty} \operatorname{E} [ ( \mathbf{z}n - \mathbf{z} )'( \mathbf{z}n - \mathbf{z} ) ] \ & = \lim{n \to \infty} \left( \operatorname{E} [ ( z{n1} - z_1 )^2 ] + \cdots + \operatorname{E} [ ( z_{nK} - z_K )^2 ] \right) \ & = \lim_{n \to \infty} \operatorname{E} [ ( z_{n1} - z_1 )^2 ] + \cdots + \lim_{n \to \infty} \operatorname{E} [ ( z_{nK} - z_K )^2 ] \ & = 0. \end{align} $$

If $$ \lim_{n \to \infty} \operatorname{E} [ ( \mathbf{z}_n - \mathbf{z} )'( \mathbf{z}_n - \mathbf{z} ) ] = 0 $$, then

$$ \lim_{n \to \infty} \operatorname{E} [ ( z_{n1} - z_1 )^2 ] + \cdots + \lim_{n \to \infty} \operatorname{E} [ ( z_{nK} - z_K )^2 ] = 0. \tag{1} $$

Because each terms in (1) are non-negative, (1) implies

$$ \lim_{n \to \infty} \operatorname{E} [ ( z_{nk} - z_k )^2 ] = 0, \text{ for $k = 1, \ldots, K$}, $$

which means $$ z_{nk} \to_{m.s.} z_k $$ for $$ k = 1, \ldots, K $$, and this implies $$ \mathbf{z}n \to{m.s.} \mathbf{z} $$ by definition in the text.

(b) Similarly, if $$ \mathbf{z}_n \to_p \boldsymbol{\alpha} $$, by definition in the text, for any $$ \varepsilon > 0 $$,

$$ \operatorname{Prob} \left( | z_{nk} - \alpha_k | > \varepsilon \right) \to 0, \text{ for $k = 1, \ldots, K$}, $$

then we can show

$$ \begin{gather} \operatorname{Prob} \left( ( z_{nk} - \alpha_k )^2 > \varepsilon^2 \right) \to 0, \text{ for $k = 1, \ldots, K$}, && \text{(equivalent event)} \

\operatorname{Prob} \left( ( z_{nk} - \alpha_k )^2 > \varepsilon \right) \to 0, \text{ for $k = 1, \ldots, K$}, && \text{(any $\varepsilon > 0$)} \

\operatorname{Prob} \left( ( z_{n1} - \alpha_1 )^2 > \varepsilon \right) + \cdots + \operatorname{Prob} \left( ( z_{nK} - \alpha_K )^2 > \varepsilon \right) \to 0, && \text{(limit sum)} \

\left( \not\Rightarrow \operatorname{Prob} \left( (z_{n1} - \alpha_1)^2 + \cdots + (z_{nK} - \alpha_K)^2 > K \varepsilon \right) \to 0 \right) && \text{(bigger event)} \

\operatorname{Prob} \left( \bigcup_{k=1}^K \left( ( z_{nk} - \alpha_k )^2 > \varepsilon \right) \right) \to 0, && \text{(smaller event)} \

\operatorname{Prob} \left( \bigcap_{k=1}^K \left( ( z_{nk} - \alpha_k )^2 \leq \varepsilon \right) \right) \to 1, && \text{(complement event)} \

\operatorname{Prob} \left( ( z_{n1} - \alpha_1 )^2 + \cdots + ( z_{nK} - \alpha_K )^2 \leq K \varepsilon \right) \to 1, && \text{(bigger event)} \

\operatorname{Prob} \left( ( z_{n1} - \alpha_1 )^2 + \cdots + ( z_{nK} - \alpha_K )^2 \leq \varepsilon \right) \to 1, && \text{(any $\varepsilon > 0$)} \

\operatorname{Prob} \left( ( \mathbf{z}_n - \boldsymbol{\alpha} )' ( \mathbf{z}_n - \boldsymbol{\alpha} ) \leq \varepsilon \right) \to 1, && \text{(matrix notation)} \

\operatorname{Prob} \left( ( \mathbf{z}_n - \boldsymbol{\alpha} )' ( \mathbf{z}_n - \boldsymbol{\alpha} )

\varepsilon \right) \to 0. && \text{(complement event)} \end{gather} $$

If $$ \operatorname{Prob} \left( ( \mathbf{z}_n - \boldsymbol{\alpha} )' ( \mathbf{z}_n - \boldsymbol{\alpha} ) > \varepsilon \right) \to 0 $$, then we can show

$$ \begin{gather} \operatorname{Prob} \left( ( z_{n1} - \alpha_1 )^2 + \cdots + ( z_{nK} - \alpha_K )^2 > \varepsilon \right) \to 0, && \text{(scalar notation)} \

\operatorname{Prob} \left( \bigcup_{k=1}^K \left( ( z_{nk} - \alpha_k )^2 > \varepsilon \right) \right) \to 0, && \text{(smaller event)} \

\operatorname{Prob} \left( ( z_{nk} - \alpha_k )^2 > \varepsilon \right) \to 0 \text{ for $k = 1, \ldots, K$ }, && \text{(smaller events)} \end{gather} $$

which means $$ z_{nk} \to_p \alpha_k $$ for $$ k = 1, \ldots, K $$, and this implies $$ \mathbf{z}_n \to_p \boldsymbol{\alpha} $$.

---

Copyright ©2018 by Qiang Gao