Sample variance

If we use sample mean: \(\frac{\sum_i{\left( X_i-\bar{X} \right)}^2}{n}\), then we should use sample variance:\(\frac{\sum_i{\left( X_i-\bar{X} \right)}}{n-1}\), mind the denominator is \(n-1\) instead of \(n\). And if we use expectation value \(E(X)\), then we should use variance formula $ \frac{\sum_i{\left( X_i-E\left( X \right) \right)}}{n}$.

Let us turn to the problem of estimating \(\sigma^{2}=\frac{\sum_i{\left( X_i-E\left( X \right) \right)}^2}{n}\),

\[\begin{aligned} S^{2} &=\sum_{i=1}^{n} \frac{\left(X_{i}-\bar{X}\right)^{2}}{n-1}=\sum_{i=1}^{n} \frac{\left(X_{i}^{2}-2 X_{i} \bar{X}+\bar{X}^{2}\right)}{n-1} \\ &=\frac{\left(\sum_{i=1}^{n} X_{i}^{2}\right)-2\left(\sum_{i=1}^{n} X_{i}\right) \bar{X}+\left(\sum_{i=1}^{n} \bar{X}^{2}\right)}{n-1} \\ &=\frac{1}{n-1}\left[\sum_{i=1}^{n} X_{i}^{2}-\frac{\left(\sum_{i=1}^{n} X_{i}\right)^{2}}{n}\right] \end{aligned} \]

Now if we calculate the average value of \(S^{2}\) we have

\[\begin{aligned} E\left(S^{2}\right) &=\frac{1}{n-1}\left\{\sum E\left[X_{i}^{2}\right]-\frac{1}{n} E\left[\left(\sum X_{i}\right)^{2}\right]\right\} \\ &=\frac{1}{n-1}\left\{\sum\left(\sigma^{2}+\mu^{2}\right)-\frac{1}{n}\left\{V\left(\sum X_{i}\right)+\left[E\left(\sum X_{i}\right)\right]^{2}\right\}\right\} \\ &=\frac{1}{n-1}\left\{n \sigma^{2}+n \mu^{2}-\frac{1}{n} n \sigma^{2}-\frac{1}{n}(n \mu)^{2}\right\} \\ &=\frac{1}{n-1}\left\{n \sigma^{2}-\sigma^{2}\right\}=\sigma^{2} \end{aligned} \]

Than we have show that the sample variance \(S^{2}\) is an unbiased estimator of \(\sigma^{2}\). The estimator that uses the divisor \(n\) can be expressed as \((n-1) S^{2} / n\), so

\[E\left[\frac{(n-1) S^{2}}{n}\right]=\frac{n-1}{n} E\left(S^{2}\right)=\frac{n-1}{n} \sigma^{2} \]

This estimator is therefore biased.