9.2 A First Example

Note that the smaller the selected value, the stronger the evidence against $H_0.$ That is, if $H_0$ is true, then smaller values are less likely to be observed. This is called the direction of extreme. The direction of extreme is identified by looking at what test statistic values provide stronger evidence against the null hypothesis. In this example, the direction of extreme is said to be to the left, because on a number line, numbers get smaller as you move to the left (at least in the manner we usually draw them).

In a hypothesis test, we must decide on what constitutes “significant evidence against $H_0$.” This choice is called a decision rule. A decision rule will be of the form, “if we observe a test statistic of this value or more extreme, then we will reject $H_0.$” For example, a decision rule could be:

1. We will reject $H_0$ if the number selected is 1 or less.

The chosen value of 1 is called a critical value.

If $H_0$ is true, then the chance of making a Type I error is called the significance level and is denoted by $\alpha .$ Recall that a Type I error occurs when $H_0$ is true, but the decision is to reject $H_0.$

A decision rule and significance level are equivalent. For example, suppose the decision rule is “reject $H_0$ if the number selected is 1 or less.” If $H_0$ is true, the chance of rejecting $H_0$ is $\alpha =1/15.$ Conversely, suppose that $\alpha =1/15.$ Then this would determine a decision rule of “reject $H_0$ if the number selected is 1 or less,” with the critical value being 1.

If $H_0$ is false, then the chance of making a Type II error is denoted by $\beta .$ Recall that a Type II error is made when $H_0$ is false, but we fail to reject $H_0.$ A decision rule or choice for $\alpha$ both determine $\beta .$ For example, suppose the decision rule is “reject $H_0$ if the number selected is 1 or less,” and suppose that $H_0$ is false. In order to fail to reject $H_0,$ a number greater than 1 must be selected. If $H_0$ is false, the chance of that happening is $\beta =10/15=2/3.$ Figure 9.2 gives a visual representation of the computation for computing $\alpha$ and $\beta$ for this decision rule.

A choice of $\alpha$ or decision rule leads to the following decision making scheme: The critical value and direction of extreme define a rejection region. If the test statistic falls within the rejection region, then $H_0$ is rejected, i.e., if the test statistic is at the critical value or more extreme, then $H_0$ is rejected.

To illustrate, suppose that we again use the decision rule of “reject $H_0$ if the number selected is 1 or less.” And suppose upon selecting the population, the number 3 is chosen as the sample. Since 3 does not fall with the rejection region, we fail to reject $H_0,$ i.e., there is insufficient evidence to reject $H_0.$ Figure 9.3 gives a visualization of the regions.

How hypothesis testing is done varies by discipline, and varies by the vintage of the hypothesis tester. The “rejection region” approach is something you may well see, but it is a bit “old fashioned.” Officially, it is called the Classical Approach. A more modern hypothesis testing scheme is as follows: Instead of picking a decision rule, it is more common that the researcher picks a value for $\alpha ,$ i.e., the researcher decides on an acceptable level of risk in making a Type I error.³⁸³⁸Recall that a Type I error cannot occur if the chosen $H_0$ is false. Since a Type I error is generally the more serious error, common choices for $\alpha$ are 0.10, 0.05 or 0.01; the more serious a Type I error is, the smaller the chosen value of $\alpha .$³⁹³⁹Even smaller values of $\alpha$ are used, for example in space flight, when an error can lead to catastrophic results. Instead of computing a critical value from the choice of $\alpha ,$ the researcher computes a $p$-value, which is discussed next.

After choosing $H_0$ and $H_1,$ picking $\alpha ,$ designing an experiment and collecting the data, and computing the test statistic from the sample, the researcher then asks:

• What is the probability of observing this test statistic, or anything more extreme, assuming $H_0$ is true?

This probability is called a p-value. The “more extreme” phrase in the $p$-value question always references the direction of extreme, just as with the decision rule and computations of $\alpha$ and $\beta .$ Computing a $p$-value is similar to computing $\alpha ,$ in that the first step is to assume $H_0$ is true.

For example, suppose that the number 3 is chosen. Then the corresponding $p$-value is computed by answering the following question: Assuming the chosen population is Population 1 ($H_0$ is true), what is the chance of observing a 3 (the test statistic) or anything smaller (more extreme)? Thus, if a 3 is chosen, then the $p$-value is 6/15, or $0.4$, since there are six numbers in Population 1 that are 3 or less. Figure 9.7 illustrates.

To make a decision, the researcher compares the computed $p$-value to the the choice for $\alpha .$ If the $p$-value is $\le \alpha ,$ then $H_0$ is rejected; otherwise one fails to reject $H_0.$ This procedure is equivalent to the decision rule scheme. Why? The $p$-value is $\le \alpha$ if and only if the test statistic falls within the rejection region.

Some researchers have started to skip using an $\alpha ,$ i.e., in reporting results, researchers simply report $p$-values, and let readers themselves interpret whether the results are significant. We won’t do that in this text, i.e., we will consistently use a choice for $\alpha$ in interpreting results.

Concepts Check

1. 1.

   If $H_0$ is false, what is the truth value of $H_1$?
   Answer: True

2. 2.

   When a researcher chooses $H_0,$ what is the probability that $H_0$ is true?
   Answer: 0 or 1

3. 3.

   If one fails to reject $H_0,$ what type of error might have been made?
   Answer: Type II, but remember that this error is possible only if $H_0$ is false

4. 4.

   The direction of extreme is identified by looking at what values of the test statistic are less likely if $H_0$ is true.
   Answer: True

5. 5.

   True or false: The $p$-value is the chance of observing the test statistic, or anything more extreme, assuming $H_0$ is true.
   Answer: True

6. 6.

   True or false: The significance level, $\alpha ,$ is the chance of making a Type I error.
   Answer: True, but remember that a Type I error is possible only if $H_0$ is true

7. 7.

   Assuming the Population 1 / Population 2 scenario above, if the number selected is 2, what is the corresponding $p$-value?
   Answer: $3/15=1/5=0.2$

8. 8.

   Assuming the Population 1 / Population 2 scenario above, if the decision rule is “reject $H_0$ if the selected number is 2 or less,” what is the value of $\alpha \mathrm{?}$
   Answer: $0.2,$ same as above

9. 9.

   Assuming the Population 1 / Population 2 scenario above, if the decision rule is “reject $H_0$ if the selected number is 2 or less,” what is the value of $\beta \mathrm{?}$
   Answer: Remember to assume $H_0$ is false, and that the voucher gotten is greater than $200, so $\beta =6/15=0.4$

10. 10.

    Suppose that in a hypothesis test, the researcher chose $\alpha =\mathrm{0.05.}$ Suppose further that after computing the test statistic, the researcher got a $p$-value of 0.0421. What is the researchers decision?
    Answer: Reject $H_0$

11. 11.

    Suppose that in a hypothesis test, the researcher chose $\alpha =\mathrm{0.05.}$ Suppose further that after computing the test statistic, the researcher got a $p$-value of 0.0882. What is the researchers decision?
    Answer: Fail to reject $H_0$