A First Course in Computational Thinking

Chapter 9 Hypothesis Testing - An Introduction

Goal:
∙ Introduce hypothesis testing.

This chapter introduces the topic of hypothesis testing. There are a lot of concepts, terminology and various computations involved with hypothesis testing, so it takes some patience to wade through it all. If you are new to this, the concepts and terminology may seem strange, but by the end of this course, have confidence that you will know these things well.

As was introduced via confidence intervals, when doing science we are confronted with uncertainty. In making a decision based on data, there is risk in making a mistake. Why? Because examining all possible data (i.e., the population) is usually impossible, most often because of size, access, time or money. We must make decisions based on partial information, inferring something about the population based on a sample. When making such an inference, we risk making an error. And, it is usually impossible to know if a mistake was made.

A essential example to illustrate is the practice of medicine. Suppose a pharmaceutical company manufactures a current drug for treating Illness A. The current drug is effective, but could be improved. The company has invented a new drug that is designed to be more effective than the current one. The company tests the new drug on a sample of subjects, and finds that for the sample, the new drug was on average more effective than the current. Consider the following questions:

  1. 1.

    Can the company be certain that the new drug is on average better than the current?

  2. 2.

    What are the risks to patients in abandoning the current drug in favor of the new?

  3. 3.

    What are the risks to the company in abandoning the current drug in favor of the new?

  4. 4.

    What are the risks in not abandoning the current drug and manufacturing the new in its place?

Hypothesis Testing is a collection of strategies used in making decisions, such as deciding whether to abandon the current drug in favor of the new. In this chapter, we will illustrate the mechanics of hypothesis testing using populations that are small and that we control, i.e., we’re going to rig the game so that we can see exactly what is happening.