What Is a Null Hypothesis?
A null hypothesis (H0) is a default statistical statement asserting that no effect, no difference, or no relationship exists between variables in a study. It serves as the starting assumption in hypothesis testing, the claim you're trying to disprove with data. For example, if you're testing whether a new product page design increases conversion rates, the null hypothesis states that the new design has no effect on conversions compared to the original. Researchers collect data and use statistical tests to determine whether the evidence is strong enough to reject H0 in favor of the alternative hypothesis (H1), which proposes that an effect or difference does exist.
Why the Null Hypothesis Matters in Research
The null hypothesis provides a structured framework for making evidence-based decisions rather than relying on intuition. Without it, there's no formal standard for distinguishing real effects from random variation in data. In A/B testing alone, companies that skip proper hypothesis formulation report false-positive rates as high as 30%, according to a 2020 analysis by Optimizely, meaning nearly one in three "winning" variants isn't actually better than the control.
How the Null Hypothesis Works
H0 vs. H1
Every hypothesis test involves a pair of competing statements:
- H0 (Null hypothesis): The status quo. No effect exists. Any observed difference is due to chance.
- H1 (Alternative hypothesis): Something is happening. An effect, difference, or relationship exists.
The alternative hypothesis can be directional or non-directional:
- Directional (one-tailed): H1 specifies the direction of the effect. Example: "The new design increases conversion rate" (not just "changes" it).
- Non-directional (two-tailed): H1 states that a difference exists without specifying direction. Example: "The new design produces a different conversion rate than the original."
You never "prove" the null hypothesis. You either reject it (the evidence suggests an effect exists) or fail to reject it (the evidence isn't strong enough to conclude an effect exists). Failing to reject H0 doesn't mean no effect exists, it means you didn't detect one with your current data.
P-Values and Decision Rules
The p-value tells you the probability of observing results at least as extreme as yours if the null hypothesis were actually true.
- A small p-value (typically < 0.05) means the observed data would be unlikely under H0, so you reject the null hypothesis.
- A large p-value means the data is consistent with H0, so you fail to reject it.
The threshold for rejection, called alpha (α), is set before data collection. The convention is α = 0.05 (5%), though some fields use stricter thresholds. Medical research often uses α = 0.01, while exploratory market research sometimes accepts α = 0.10.
Here's what this looks like in practice:
Research question: Does offering free shipping increase average order value?
H0: Free shipping has no effect on average order value (μ_free = μ_standard).
H1: Free shipping increases average order value (μ_free > μ_standard).
Result: After running the experiment, you calculate a p-value of 0.03. Since 0.03 < 0.05, you reject H0 and conclude that free shipping likely increases average order value.
Type I and Type II Errors
Hypothesis testing involves two kinds of mistakes:
| Error Type | What Happens | Also Called | Controlled By |
|---|---|---|---|
| Type I | Rejecting H0 when it's actually true | False positive | Alpha level (α) |
| Type II | Failing to reject H0 when it's actually false | False negative | Statistical power (1-β) |
Type I error is concluding that an effect exists when it doesn't. If you set α = 0.05, you accept a 5% risk of this error. In business terms, this means launching a product change that doesn't actually improve anything.
Type II error is missing a real effect. This happens when your sample size is too small or the effect is subtle. In business terms, this means killing a product change that would have worked.
The two errors trade off against each other. Making α stricter (harder to reject H0) reduces Type I errors but increases Type II errors. The way to reduce both simultaneously is to increase your sample size.
Statistical Power
Power is the probability of correctly rejecting a false null hypothesis. It depends on three factors: sample size, effect size, and alpha level. Most researchers aim for power of 0.80 (80% chance of detecting a real effect if one exists). A power analysis before data collection tells you how many observations you need, running it afterward is too late to fix an underpowered study.
When to Use a Null Hypothesis
- A/B testing web pages, email subject lines, or ad creatives where you need to know if the variation genuinely outperforms the control
- Product research comparing customer satisfaction scores between segments or time periods
- Pricing studies testing whether a price change affects purchase intent or willingness to pay
- Survey experiments evaluating whether different question framings produce different response patterns
- Quality assurance checking whether a process change affected output metrics
Common Mistakes to Avoid
- Interpreting "fail to reject H0" as proof that no effect exists: absence of evidence isn't evidence of absence, especially with small samples
- Setting alpha after seeing results (p-hacking), the decision threshold must be established before data collection
- Ignoring effect size and focusing only on statistical significance, a statistically significant result can be too small to matter practically
- Running multiple tests without correction: testing 20 variables at α = 0.05 means you'd expect one false positive by chance alone
- Confusing correlation with causation: rejecting H0 in an observational study shows association, not necessarily a causal relationship
How Quali-Fi Supports Hypothesis Testing
Quali-Fi's platform includes built-in statistical testing across all plan tiers, automatically flagging significant differences in cross-tabulated survey results. The Research plan ($1,061/month) adds sample size calculators and power analysis tools so you can determine the right number of responses before launching a study. For complex experimental designs, conjoint analysis, MaxDiff, Van Westendorp pricing, the platform handles the statistical modeling and presents results with confidence intervals and significance indicators.
Frequently Asked Questions
What's the difference between a null hypothesis and a research hypothesis?
A research hypothesis is your prediction about what you expect to find, it aligns with the alternative hypothesis (H1). The null hypothesis is the opposite: the default position that your prediction is wrong. You test the null hypothesis statistically, and if the data contradicts it strongly enough, you accept your research hypothesis.
Can a null hypothesis be proven true?
No. Statistical testing can only reject or fail to reject the null hypothesis. Failing to reject it means the data doesn't provide sufficient evidence against it, but that's not the same as proving it's true. A larger sample, a different measurement approach, or a more sensitive test might detect an effect that your current study missed.
Why do we test the null rather than testing the alternative directly?
The logic comes from falsification in the philosophy of science. It's easier to disprove a specific claim ("there is no difference") than to prove an open-ended one ("there is a difference of some unknown size"). By assuming no effect exists and looking for evidence to the contrary, the framework provides a clear decision rule and a quantifiable error rate.