In the Physicians' Reactions study, the researchers hypothesized that physicians would expect to spend less time with obese patients. The null hypothes is that the two types of patients are treated identically is put forward with the hope that it can be discredited and therefore rejected. So the null hypotheis is
H0: μobese = μaverage
In the physician reaction study, we compute the probability of getting a difference as large or larger than the observed difference (31.4 - 24.7 = 6.7 minutes) if the difference were, in fact, due solely to chance. This probability can be computed to be 0.0057. Since this is such a low probability, we have confidence that the difference in times is due to the patient's weight and is not due to chance.
The probability value below which the null hypothesis is rejected is called the α level or simply α. It is also called the significance level. When the null hypothesis is rejected, the effect is said to be statistically significant. It is very important to keep in mind that statistical significance means only that the null hypothesis of exactly no effect is rejected; it does not mean that the effect is important. Do not confuse statistical significance with practical significance.
Two ways of significance tests
- A significance test is conducted and the probability value reflects the strength of the evidence against the null hypothesis. Higher probabilities provide less evidence that the null hypothesis is false. (For scientific research)
|p<0.01||The data provide strong evidence that the null hypothesis is false.|
|0.01<p<0.05||The null hypothesis is typically rejected, but not with as much confidence as it would be if the probability value were below 0.01.|
|0.05<p<0.1||The data provide weak evidence against the null hypothesis and are not considered low enough to justify rejecting it.|
- Specify an α level before analyzing the data. If the data analysis results in a probability value below the α level, then the null hypothesis is rejected; if it is not, then the null hypothesis is not rejected. If a result is significant, then it does not matter how significant it is.
If it is not significant, then it does not matter how close to being significant it is.
Type I and II Errors
Type I error (弃真错误) occurs when a significance test results in the rejection of a true null hypothesis. α is the probability of a Type I error given that the null hypothesis is true.
Type II error (弃伪错误) is failing to reject a false null hypothesis. If the null hypothesis is false, then the probability of a Type II error is called β (beta). The probability of correctly rejecting a false null hypothesis equals 1- β and is called power. Actually, a Type II error is not really an error. When a statistical test is not significant, it means that the data do not provide strong evidence that the null hypothesis is false. Lack of significance does not support the conclusion that the null hypothesis is true. One way to decrease the value of β is to increase the volume of samples. With the constance volume of samples, β will increase with smaller value of α. In practice, we should perform a trade of between α and β.
One- and Two-Tailed Tests
Whether it's a one-tailed test or two-tailed test depends on the way the question is posed. If we are asking whether physicians spend different time with obese patients, then we would conclude they do if they spent either much more than chance or much less than chance. So the null hypothesis for the two-tailed test is
H0: μobese = μaverage
If our question is whether physicias spend less time with obese patients, we would use a one-tailed test and the null hypothesis is
H0: μobese ≥ μaverage
Significance Testing and Confidence Intervals
- The 95% confidence interval corresponds to 0.05 significance level. The 99% confidence interval corresponds to 0.01 significance level.
- Whenever an effect is significant, all values in the confidence interval will be on the same side of zero. Therefore, a significant finding allows the researcher to specify the direction of the effect.
- If the 95% confidence interval contains zero (more precisely, the parameter value specified in the null hypothesis), then the effect will not be significant at the 0.05 level. That is why the null hypothesis should not be accepted when it is not rejected.