What Is Level of Significance?
The level of significance α is the probability of rejecting a true null hypothesis, called a Type I error. Common choices are 0.10, 0.05 and 0.01, but the value should be chosen before you look at the data, based on the cost of a false positive.
Enter α and choose a one-tailed or two-tailed test to get the critical value and the tail area in each rejection region. Add an effect size and sample size to see the power (1 − β) and the Type II error probability. The page also shows how many false positives to expect across many tests and the Bonferroni-adjusted α for 10 tests.
Below you will find the formulas, a variables table, a worked example and answers to common questions such as significance level versus p-value.
How to Use the Level of Significance Calculator
- Select α and whether the test is one- or two-tailed.
- Choose z or t; for t, enter the degrees of freedom.
- Optionally enter an effect size d and sample size n to see power and Type II error.
- Read the critical values and use them to decide a test: reject H₀ if your statistic falls in the shaded region.
Level of Significance Formula
Significance level
Two-tailed critical value
Power (two-tailed z test)
Level of Significance Variables Explained
| Symbol | Meaning | Notes |
|---|---|---|
| α | Significance level | Type I error rate |
| zα | Critical value | Cut-off for the rejection region |
| β | Type II error probability | Failing to detect a real effect |
| 1 − β | Power | Chance of detecting a real effect |
Level of Significance Worked Example
Using the example values (press “Load example” above to try them):
- α = 0.05 is the probability of rejecting a true null hypothesis (Type I error).
- For a two-tailed z test, the critical value is ±1.96.
- Noncentrality = d√n = 2.828; power = 0.8074, so β = 0.1926.
Result: Significance level α = 0.05; Critical values (±) = ± 1.96; Power (1 − β) = 80.74 %.
Type I and Type II errors
The two errors trade off for a fixed sample size: demanding stronger evidence (smaller α) makes a Type II error more likely. The only way to improve both at once is to collect more data or reduce noise.
| H₀ true | H₀ false | |
|---|---|---|
| Reject H₀ | Type I error (probability α) | Correct: power (1 − β) |
| Fail to reject H₀ | Correct (1 − α) | Type II error (probability β) |
How to choose α
Choose α based on the consequences of a false positive and decide it in advance. Picking α after seeing the p-value is a form of p-hacking.
| α | Confidence | Typical use |
|---|---|---|
| 0.10 | 90% | Exploratory work, early screening |
| 0.05 | 95% | Standard in most sciences and business |
| 0.01 | 99% | When false positives are costly, such as medical claims |
| 0.001 | 99.9% | Very high-stakes or many comparisons |
| 5 × 10⁻⁷ | 5 sigma | Particle physics discovery claims |
Multiple comparisons
If you run m independent tests at α = 0.05 each, the chance of at least one false positive is 1 − 0.95^m, which is about 40% for 10 tests. The Bonferroni correction divides α by m; the results show the corrected value for 10 tests. Less conservative procedures such as Holm or Benjamini-Hochberg are used when many tests are run.
Power and sample size
Power is the probability of correctly rejecting a false null hypothesis. Conventional targets are 80% or 90%. The chart shows how power climbs with sample size for your chosen effect size. Small effects need very large samples: detecting d = 0.2 at 80% power needs roughly 200 observations per group in a two-sample test, compared with about 25 for d = 0.8.
Where Level of Significance Is Used
Choosing alpha
Seeing the critical values that go with 0.10, 0.05 or 0.01.
Understanding errors
Separating Type I and Type II error and how power relates.
Multiple testing
Seeing how many false positives to expect and the Bonferroni α.
Coursework
Checking critical values and rejection regions.
Common Level of Significance Mistakes to Avoid
- Set α before looking at the data.
- α = 0.05 is a convention, not a law of nature.
- Report exact p-values rather than only “significant / not significant”.
- Check power before running a study, not after.
Level of Significance Terms and Definitions
- Rejection region
- The set of test statistic values that lead to rejecting H₀.
- Critical value
- The boundary of the rejection region.
- Power
- The probability a test detects a true effect of a given size.
Level of Significance FAQs
What is the level of significance?
The probability threshold, α, for rejecting the null hypothesis. It equals the Type I error rate you are willing to accept.
Why is 0.05 the usual significance level?
It is a long-standing convention that balances false positives against missed effects in many fields. It is not a theoretical optimum.
What is the difference between significance level and p-value?
The significance level α is a threshold you choose beforehand. The p-value is computed from your data and compared with it.
What is the critical value for 0.05?
For a two-tailed z test it is ±1.96; for a one-tailed test it is 1.645.
How does α relate to confidence level?
Confidence level = 1 − α. A 5% significance level corresponds to a 95% confidence level.
What is statistical power?
The probability of detecting an effect that really exists. It equals 1 minus the Type II error rate.
Does a smaller α always make a test better?
No. It reduces false positives but increases false negatives unless the sample size grows.