Errors in Hypothesis Testing


There are two basic types of errors that can occur in hypothesis testing:

  • Type A or 1 Error: The null hypothesis is correct, but is incorrectly rejected.
  • Type B or 2 Error: The null hypothesis is incorrect, but is not rejected.

The traditional way of explaining testing errors is with a table like the one shown below:


Typically, we’re more worried about Type A errors than Type B – rejecting a hypothesis that is not actually incorrect.

The chance of making a Type A error is referred to as the alpha risk or alpha level; the chance of making a Type B error is referred to as the beta risk or beta level.

Need more explanation? Khan Academy’s video does a good job of walking through Type A (or Type 1) errors:

ASQ Six Sigma Green Belt Errors in Hypothesis Testing Questions:

Question: When an inspection process rejects conforming product, what type of error is being made?

(A) a
(B) b
(C) σ
(D) H0

Answer: (A) An alpha error is made when you reject the null hypothesis when it is actually true. In this case, the null is that the product conformed. See errors in hypothesis testing.

A Beta error is when you fail to reject the null when the null is false. I this case, a beta error would be made if the inspection process accepted a conforming product.

The sigma symbol has nothing to do with error types. It refers to standard deviation.

The H0 symbol stands for Null Hypothesis, which is not an error type.  Also see the guide to hypothesis testing.

Question: Which of the following terms is used to describe the risk of a type I error in a hypothesis test?

(A) Power
(B) Confidence level
(C) Level of significance
(D) Beta risk

Answer: (C) Level of significance. The null hypothesis is rejected if the p-value is less than the significance or α level. The α level is the probability of rejecting the null hypothesis given that it is true (type I error) and is most often set at 0.05 (5%).

We can reject Power as it makes no sense.

A confidence level refers to the percentage of all possible samples that can be expected to include the true population parameter. It is a type of interval estimate of a population parameter. Does not makes sense here.

Beta risk is a type 2 error, so we can ignore that option. Also see the guide to hypothesis testing.

Question: Which of the following is a commonly accepted level for alpha risk?

(A) 0.05
(B) 0.50
(C) 0.70
(D) 0.95

Answer: (a) 0.05 – Remember that alpha risk is the risk that we have incorrectly rejected the null hypothesis even though it is correct, one of the kinds of errors we can make. 0.05 alpha means that we are comfortable with a 5% chance of incorrectly rejecting the null. That is a common industry standard. Even if you didn’t know that, the other answers don’t make sense. At 50% you are basically flipping a coin! Also see the guide to hypothesis testing.

1 comment… add one
  • Tim Harrington

    Very helpful, a very good resourse

Leave a Comment