A p-value is used to decide whether hypothesis test results are statistically significant or not. Once a p-value is calculated from analyzing test data, it is compared to the selected alpha level – if lower than the alpha level, the results are deemed to be statistically significant; if higher, the results are deemed to not be statistically significant.
A p-value is expressed as a number between 0 and 1.
This Khan Academy video explains p-values, and how they apply to hypothesis tests, in-depth:
While the p-value is a standard method of determining this key measurement of one’s results, there are a mix of opinions on whether it’s actually the ideal solution.
Geoff Cumming—Emeritus Professor at La Trobe University in Melbourne, Australia—explains his distrust of p-values in hypothesis testing in this video:
But then, Jeff Leek and Rafa Irizarry disagree, and their article lists a number of useful properties of the p-value. To summarize (the full list, with explanations, is available in the linked article):
- They’re easy to calculate.
- They’re easy to understand.
- They have simple, universal properties.
- Their calibration is within useful error rates.
- They can be used in correction of multiple tests.
- They’re reproducible.