8.1 Confidence Intervals for the Mean

Recall that a $Z$-score is a random variable that is normally distributed with mean 0 and standard deviation 1. That is, $Z\sim N(0,1)$. The critical value $z^{*}$ is an outcome from a $Z$-score that has the following relation to $\alpha$.

Figure 8.1 shows a representation of what $z^{*}$ stands for on a standard normal distribution. Since the standard normal distribution is symmetric about 0, there is an equal and opposite value, $-z^{*}$ that stands for $P(Z\le -z^{*})=\alpha /2$. If you add these two areas together, you get a combined area of $\alpha$. Thus, the area between $-z^{*}$ and $z^{*}$ is $1-\alpha$, the confidence level we intend on having, since the total area under the standard normal density curve is 1. How does $P(-z^{*}\le Z\le z^{*})=1-\alpha$ help give us a confidence interval for $\mu$? Since we have the relation

with a bit of algebra, we can manipulate the inequality within the probability statement to look like

Extracting the inequality from within the parentheses and replacing the random variable $\overline{X}$ with an outcome $\overline{x}$, we have the confidence interval desired. For further explanation, see Derivation of Confidence Interval for $\mu$, $\sigma$ Known at the end of this section.

So obtaining the critical value, $z^{*}$, is necessary in order to construct a $1-\alpha$ confidence interval for $\mu$. Let’s look into how to find $z^{*}$. In Excel and Python, areas under probability density curves are calculated from the left. If we want the area under the standard normal density curve to the right of $z^{*}$ to be $\alpha /2$, then the area to the left of $z^{*}$ would be $1-\alpha /2$. The commands in Figure 8.2 reflect this along with reminding you of the commands used to calculate $z^{*}$ in Excel and Python.

Let’s practice finding $z^{*}$ in both Excel and Python.

There are a few things that are required first in order to construct the $1-\alpha$ confidence interval for $\mu$, knowing $\sigma$:

Be sure you have the following:

• •

  Make sure your unbiased random sample is coming from a normally distributed population or the sample size is large enough ($n\ge 30$).³³³³It is always a good idea to assess normality first.

• •

  Make sure you have $\sigma$. Is it given to you?

• •

  Calculate the sample mean, $\overline{x}$, unless it is given to you.

• •

  Based on your confidence level, $1-\alpha$, calculate the critical value $z^{*}$.

The following table recaps the commands in Excel and Python that are necessary for computing the confidence interval for the mean, with $\sigma$ known. Each row stands for equivalent commands.

Let’s see a few examples of constructing confidence intervals for $\mu$, $\sigma$ known, using Excel and Python.

You may wonder: How does this formula come to be? Recall from the Central Limit Theorem that for a sufficiently large sample size $n$, the distribution of sample means, $\overline{X}$, becomes normal. That is,

Knowing this tells us the likelihood of obtaining different sample means. For standardization purposes, we can transform $\overline{X}$ into a $Z$-score by using the following formula:

and, as a result, the random variable $Z$ is normally distributed with $\mu =0$ and $\sigma =1$, or, rather

We want $1-\alpha$ to be the likelihood that the $Z$-score, obtained from the sample mean calculated, lies between $-z_{\alpha /2}$ and $z_{\alpha /2}$, which are just $Z$-scores that correspond to $P(Z\le -z_{\alpha /2}=\alpha /2$ and $P(Z\ge z_{\alpha /2})=\alpha /2$, respectively. This means that

By replacing $Z$ with ${\displaystyle \frac{\overline{X}-\mu }{\sigma /\sqrt{n}}}$ and manipulating the inside inequality, we have

Flipping the inside inequality around, we have

Remember that $\overline{X}$ is a random variable and $\overline{x}$ is an outcome from $\overline{X}$. Equation 8.1 states that out of all confidence intervals we can construct of the form

$(1-\alpha )$% of them will capture the population mean, $\mu$. Hence, we have obtained our desired confidence interval.

The random variable $T$ has a $t$-distribution with degrees of freedom $n-1$. The critical value, $t^{*}$ is a particular outcome of $T$ that has the following relation to $\alpha$.

The $t$-distribution is also symmetric about 0, similar to that of standard normal distribution. So the symmetric value, $-t^{*}$, stands for $P(T\le -t^{*})=\alpha /2$. Hence $P(-t^{*}\le T\le t^{*})=1-\alpha$, and thus showing the connection to the $1-\alpha$ confidence level we desire out of the confidence interval.

Obtaining $t^{*}$ is definitely necessary in order to construct a $1-\alpha$ confidence interval for $\mu$, not knowing $\sigma$. In order to find $t^{*}$, we need the degrees of freedom. Degrees of freedom is easy to calculate. Just remember,

where $n$ is the sample size taken. When calculating $t^{*}$ we will need to supply the degrees of freedom into the commands. Below the commands used to calculate $t^{*}$ in $E$ and Python.

Do you notice the similarity in the commands to the commands for finding $z^{*}$? Can you reason out why we are still using $1-0.5*\alpha$? Also notice that second input in both commands involves telling the programs what the degrees of freedom is. In both cases, $n$ stands for the sample size.

Let’s practice finding $t^{*}$ in both Excel and Python.

As in the case when we were constructing a $1-\alpha$ confidence interval for $\mu$ with $\sigma$ known, we need to make sure we have a few things before proceeding with constructing a $1-\alpha$ confidence interval for $\mu$, not knowing $\sigma$:

Be sure you have the following:

• •

  Make sure your unbiased random sample is coming from a normally distributed population.

• •

  The sample size, $n$. The sample size may be small in this case ().³⁴³⁴Extremely large sample sizes allow for one to use $z^{*}$ instead of $t^{*}$ due to the $t$-distribution becoming standard normal for incredibly large sample sizes.

• •

  You should not have $\sigma$ in this case. Use $s$, the sample standard deviation, as an estimate of $\sigma$. You may need to calculate it.

• •

  Calculate the degrees of freedom. This is $n-1$, where $n$ is the sample size.

• •

  Based on your confidence level, $1-\alpha$ and degrees of freedom, calculate the critical value $t^{*}$ using appropriate commands in either Excel or Python.

As a recap, here are the commands you will encounter in Excel and Python for constructing a $1-\alpha$ confidence interval for $\mu$, with $\sigma$ not known.

Let’s revisit Examples 8.1.3 and 8.1.4 but for the case that we do not know $\sigma$.

Comparing the confidence intervals with Examples 8.1.3 and 8.1.4 with those in Examples 8.1.7 and 8.1.8, you will notice that the confidence intervals in Examples 8.1.7 and 8.1.8 are wider. As mentioned before, estimating $\sigma$ with $s$ means we need to account for more variability. So we expect wider confidence intervals whenever we use $t^{*}$ instead of $z^{*}$.

It is important to understand what the confidence interval is trying to describe to us. First and foremost, you should remove the following from your belief of what the confidence interval is.

The true mean $\mu$ is always within the range represented by the confidence interval.

This may not be true. Confidence intervals have a likelihood involved. In fact, the confidence level $1-\alpha$ is the likelihood associated with the confidence interval.

Let’s get a better grasp of what is going on. Let’s create a $1-\alpha$ confidence interval for $\mu$, assuming the data we gather come from a normally distributed population. Assume further that $\sigma$ is known and the sample size $n$ is fixed.

We have everything needed to start calculating the $1-\alpha$ confidence interval for $\mu$ except for the sample mean. Well, that’s easy to obtain. We take a simple random sample from the normally distributed population and calculate a sample mean, call it ${\overline{x}}_1$. We have everything we need. So, we can go ahead and calculate a $1-\alpha$ confidence interval for $\mu$.

Meanwhile, a friend of ours from class also does the same thing. He or she takes a simple random sample from the population and calculates a sample mean, call it ${\overline{x}}_2$. Then they calculate a $1-\alpha$ confidence interval for $\mu$. (Note that their $\alpha$, $\sigma$,$t^{*}$, and $n$ are all the same.) Their confidence interval looks like

What’s the difference? Well, their ${\overline{x}}_2$ will most likely be different than ours. So their $1-\alpha$ confidence interval for $\mu$ will most likely be different. An argument ensues over which interval has $\mu$ in it. As a result, we both decide to repeat the process. We obtain a different sample mean, ${\overline{x}}_3$, and our obstinate friend gets a different sample mean, ${\overline{x}}_4$. Here are the different $1-\alpha$ confidence intervals for $\mu$.

These are different than our originals! Is there something wrong with the way the intervals are calculated? Nope! Remember that $\overline{x}$ changes with each simple random sample collected from a population. If our friend and us each did this 50 times for a combined total of 100 different confidence intervals, then we should notice that some of the intervals may overlap. In fact, we should expect the proportion of $1-\alpha$ of them to overlap. This is what the confidence interval is telling us.

For example, suppose that $1-\alpha =95\%$. Then if we were to construct one hundred $95\%$ confidence intervals for $\mu$, 95 of them should contain the true population mean $\mu$.

So, as a result, we should interpret the $1-\alpha$ confidence interval for $\mu$ by stating the following.

We are $1-\alpha$ confident that the confidence interval we obtained contains the population mean, $\mu$.

In the statement, we are placing emphasis on the fact that there is a chance the confidence interval we obtained may not contain the population mean, $\mu$. The above interpretation of a $1-\alpha$ confidence interval for $\mu$ can be extended to the case of when we do not know $\sigma$. In fact, this same interpretation can be extended further to a $1-\alpha$ confidence interval for any parameter, not just $\mu$.

Based on this understanding of the confidence interval, your first thought may be, “Why not make $\alpha$ as small as possible, say $\alpha =0.000001$?” This is a good question. The smaller $\alpha$ is, the more likely a constructed $1-\alpha$ confidence interval will capture the population mean, $\mu$. However, there is a drawback to making $\alpha$ be super small. The smaller $\alpha$ gets, the range the $1-\alpha$ confidence interval describes will widen. The best way to see this is through an example.

Ideally, we $\alpha =0.05$ or $\alpha =0.01$ will suffice. However, if targeting the population mean is important, there is another tactic we can take to try to shrink the confidence interval.

While keeping $\alpha =0.05$ is ideal, we can still adjust the confidence interval by adjusting the sample size $n$. That will allow us to control the width of the sample size.

Recall that the confidence interval is sometimes written as

where $E=z^{*}\frac{\sigma }{\sqrt{n}}$ or $E=t^{*}\frac{s}{\sqrt{n}}$, depending on whether $\sigma$ is known or not, this is called the margin of error for $\overline{X}$. The margin of error tells us how much we are allowed to deviate to the left and to the right of the sample mean when constructing the $1-\alpha$ confidence interval for $\mu$.

Say we want the margin of error to be not more than $\pm 0.01$. That is, whenever we construct a $1-\alpha$ confidence interval for $\mu$ we want the interval to look like

for some $\overline{x}$ that was taken from a sample. How do we insure that the margin of error is $\pm 0.01$ or rather any other desired value of margin of error?

Since $\alpha$ is assumed to be fixed and $\sigma$ is given, the only other variable we can adjust is the sample size, $n$. With a little algebra, we can obtain an expression that will tell us how big $n$ needs to be. In fact, by letting $c=z^{*}$ (or $c=t^{*}$), and $v=\sigma$ (or $v=s$), and solving the formula

for $n$, we have

This then gives us what we need.

Let’s see an example in Python for determining the appropriate sample size.