Statistical Inference I & II: Proof

1. Unbiased estimation

1.2 Population mean estimator

The estimator

\[{\bar X = \frac{1}{n} \sum_{i=1}^n X_i} \]

of the samples is a unbiased estimator of the mean of the population \({\mu=\frac{1}{N} \sum_{i=1}^N X_i}\).

That is to proof \({\mathrm E[\bar X] = \mu}\). Known \({\mathrm E[X_i] = \mu}\).

\[\begin{aligned} \mathrm E[\bar X] = \mathrm E \left[ \frac{1}{n} \sum X_i \right] = \frac{1}{n} \sum \mathrm E[X_i] = \frac{1}{n} \sum \mu = \mu \end{aligned} \]

Notes: Actually, if a population \({X}\) has \({k}\) order central moment \({\mu_k=\mathrm E[X^k](k \geq 1)}\), and \({X_1,\cdots,X_n}\) are the samples drawn from the population \({X}\). No matter which kind of the distribution of \({X}\) is, the estimator

\[{A_k = \frac 1n \sum \limits_{i=1}^n x_i^k} \]

always is the unbiased estimator of the central moment \({\mu_k}\) of the population.

1.2 Population variance estimator

The estimator

\[{s^2 = \frac{1}{n-1} \sum_{i=1}^n(X_i-\bar X)^2} \]

of the sample is a unbiased estimator of the variance of the population \({\sigma^2= 1/N \sum_{i=1}^N (X_i-\mu)^2}\).

That is to proof \({\mathrm E(s^2) = \sigma^2}\).

Know: \({\mathrm E[X_i] = \mu}\), \({\mathrm{D}[X_i]=\sigma^2}\) and \({\mathrm D[X_i] = \mathrm E[X_i^2] - \mathrm E[X_i]^2}\)

\[\begin{align} \mathrm E(s^2) &= \mathrm E \left[ \frac{1}{n-1} \sum(X_i-\bar X)^2 \right] = \frac{1}{n-1} \mathrm E \left[\sum(X_i-\bar X)^2 \right] \\ &= \frac{1}{n-1} \mathrm E \left[\sum (X_i^2 - 2 \bar X X_i + \bar X^2) \right] = \frac{1}{n-1} \left( \mathrm E \left[\sum X_i^2 - \sum 2 \bar X X_i + \sum \bar X^2 \right] \right) \\ &= \frac{1}{n-1} \left( \mathrm E \left[\sum X_i^2 - 2 \bar X \sum X_i + n \bar X^2 \right] \right) \\ &= \frac{1}{n-1} \left( \mathrm E \left[\sum X_i^2 - n \bar X^2 \right] \right) \quad \left(\because\sum X_i= n \bar X \right) \\ &= \frac{1}{n-1} \left( \sum \mathrm E [X_i^2] - n \mathrm E [\bar X^2] \right) \\ &= \frac{1}{n-1} \left( n (\sigma^2 + \mu^2)- n \left( \frac1n \sigma^2 + \mu^2 \right)\right) \\ &= \sigma^2 \end{align} \]

where \({\mathrm E [\bar X^2]}\) is calculated as:

\[\mathrm E [\bar X^2] = \mathrm D [\bar X] + \mathrm E [\bar X]^2 = \mathrm D \left[\frac1n \sum X_i \right] + \left( \mathrm E \left[\frac1n \sum X_i \right] \right)^2 = \frac1{n^2} \sum \mathrm D[X_i] + \left( \frac1n \sum \mathrm E[X_i] \right)^2 = \frac1{n} \sigma^2 + \mu^2 \]