Small Expected Counts: Chi-Square or Fisher’s Exact Test?

The research question

A community health team surveys 40 adults who recently attended a preventive-health workshop. Each participant is classified on two categorical variables:

• Follow-up screening completed: Yes or No
• Workshop format: Online or In-person

The research question is:

Is workshop format associated with whether participants complete the recommended follow-up screening?

The null hypothesis for a two-way contingency table states that there is no association between the row and column variables. In the single-sample formulation, this means that the two categorical variables are independent (Moore et al., 2021; Devore & Berk, 2012).

Step 1: Start with the observed frequencies

The synthetic observed table is:

The corresponding completion percentages are:

• Online: 1/20 = 5%
• In-person: 7/20 = 35%

Descriptively, screening completion is therefore 30 percentage points higher in the in-person group.

That difference is useful for understanding the pattern, but it is not itself the input to the Pearson chi-square calculation. Meier et al. (2014) explicitly distinguish percentaged contingency tables from the raw frequencies required for calculating expected frequencies and chi-square: if a table has been converted to percentages, the analysis must return to the raw frequencies. Moore et al. (2021) likewise define the chi-square statistic in terms of observed and expected counts.

Step 2: What would independence predict?

Under the null hypothesis of independence, the expected count for a cell is calculated from its row total, column total, and the total sample size:

Expected count = (row total × column total) / total n

(Moore et al., 2021; Devore & Berk, 2012).

For example, the expected number of online participants who complete screening is:

(20 × 8) / 40 = 4

Applying the same calculation to all four cells gives:

These expected frequencies describe the table we would anticipate, given the observed margins, if workshop format and screening completion were independent. The chi-square procedure assesses how far the observed table departs from that no-association pattern (Moore et al., 2021; Meier et al., 2014).

Here, the largest substantive departures are easy to see. Only one online participant completed screening when four were expected under independence, whereas seven in-person participants completed screening when four were expected.

Step 3: What does the Pearson chi-square statistic measure?

The Pearson chi-square statistic combines the discrepancies between observed and expected cell counts:

χ² = Σ (Observed − Expected)² / Expected

A larger value indicates greater disagreement between the observed table and the table expected under the null hypothesis. The reference chi-square distribution provides an approximate null distribution for this statistic (Moore et al., 2021; Lovric, 2011).

For the synthetic table, the Pearson calculation without a continuity correction gives:

χ²(1) = 5.625, p = .0177

If the approximation were appropriate, this result would provide evidence against independence.

But the numerical p-value is not the first decision to make. The expected counts need to be inspected before relying on that approximation.

Step 4: Check the expected counts before trusting chi-square

The four expected counts are:

4, 16, 4, 16

Two cells therefore have expected counts of only 4.

Devore and Berk (2012) similarly connect the adequacy of chi-square procedures to expected frequencies and illustrate checking estimated expected counts before relying on the chi-square analysis. Lovric (2011) states more generally that the chi-square null distribution is an approximation whose adequacy depends on expected cell frequencies not being too small.

For this case, the important diagnostic finding is therefore not simply that the sample contains 40 people. It is that the table contains sparse expected cells.

That distinction matters. A seemingly reasonable overall sample size can still produce a sparse contingency table when one outcome is uncommon or observations are distributed unevenly among categories.

Why sparse expected counts are a problem

The usual Pearson chi-square p-value relies on an approximate chi-square reference distribution. When expected cell frequencies are small, that approximation can be inadequate (Lovric, 2011; Moore et al., 2021).

This is why researchers should not decide that a chi-square test is appropriate merely because both variables are categorical. The sequence should instead be:

1. Identify the categorical variables.
2. Construct the contingency table.
3. Work from the observed counts.
4. Calculate the expected counts.
5. Assess the adequacy of the chi-square approximation.
6. Choose the test.

In this synthetic example, the expected-count check raises a problem before the Pearson result is interpreted as the primary inferential result.

Step 5: Fisher’s exact test

Fisher’s exact test is specifically developed for testing independence in a 2 × 2 contingency table. Conditional on the marginal totals, the method uses the hypergeometric distribution to evaluate possible tables under the independence hypothesis rather than relying on the large-sample chi-square approximation (Lovric, 2011).

Moore et al. (2021) note that Fisher’s exact test is preferred by many analysts when counts are small and the chi-square approximation is not sufficiently accurate.

For the synthetic table:

The two-sided Fisher exact calculation gives:

Fisher’s exact p = .0436

At a conventional .05 significance level, this synthetic result provides evidence against the null hypothesis of independence. In the sample, workshop format and screening completion are associated.

Notice that the Pearson chi-square and Fisher results are not numerically identical:

That difference is precisely why the expected-count diagnostic matters. Choosing a procedure should precede interpretation of whichever p-value happens to be smaller.

Step 6: Statistical significance is not the size of the association

A significance test answers whether the data provide evidence against the null hypothesis; it does not by itself describe the magnitude or practical importance of the relationship. Meier et al. (2014) explicitly caution against treating chi-square as a measure of relationship strength and recommend supplementing it with descriptive comparisons and measures of association.

The most immediately interpretable descriptive contrast in this case is:

35% completion for in-person participants versus 5% for online participants—a 30-percentage-point difference.

Cramér’s V is also a nominal-variable measure of association based on chi-square. Meier et al. (2014) define it as:

V = √[χ²/(mN)]

where m is the smaller of rows − 1 and columns − 1. For a 2 × 2 table, m = 1.

Using the uncorrected Pearson χ² value for this synthetic table:

V = √(5.625/40) = 0.375

Cramér’s V ranges from 0, indicating no relationship, to 1, indicating a perfect relationship for nominal variables (Meier et al., 2014). Rather than assigning an unsupported universal label such as “small,” “medium,” or “large,” the safer interpretation here is that V = .375 quantifies a nonzero sample association whose substantive importance must be judged in the health-study context.

The percentages remain important because they show the direction and practical form of the relationship, which a nondirectional measure such as Cramér’s V does not convey.

What should the researcher report?

For this synthetic study, a concise interpretation could be:

Screening completion was observed for 35% of participants attending the in-person workshop and 5% of participants attending online, a difference of 30 percentage points. Expected frequencies under independence were 4, 16, 4, and 16. Because the 2 × 2 table did not satisfy the expected-count condition described by Moore et al. (2021) for the Pearson chi-square approximation, Fisher’s exact test was used for primary inference. The two-sided Fisher test indicated evidence of an association between workshop format and screening completion (p = .044). Cramér’s V calculated from the Pearson statistic was .375, while the observed percentages showed that completion was higher in the in-person group.

This reporting strategy keeps four distinct questions separate:

(Meier et al., 2014; Moore et al., 2021; Lovric, 2011).

Independence also concerns the study design

“Independence” has more than one role in this analysis. The null hypothesis asks whether the two categorical classifications are independent in the population, but the data-generating design must also justify treating observations as separate units of information.

The standard two-way-table framework described by Moore et al. (2021) includes designs based on independent random samples or a single random sample whose observations are classified on two categorical variables. Devore and Berk (2012) similarly formulate the independence test using sampled individuals classified once on each of two factors.

Common mistake: analyzing percentages instead of frequencies

Suppose the researcher entered this table into software:

Those numbers reproduce the observed percentages, but they falsely make the table look as if it contained 200 observations rather than 40.

The chi-square statistic depends on frequencies and sample size. Meier et al. (2014) explicitly state that expected frequencies must be calculated from raw figures rather than a percentaged table. Percentages are descriptive transformations of the counts; they do not preserve the sample-size information required for the test.

Limitations of the synthetic conclusion

1. This is a deliberately small synthetic example. The result demonstrates a decision process; it does not establish an empirical relationship between workshop format and health behavior.

2. Fisher’s exact test does not repair weaknesses in sampling, measurement, or research design. It addresses the inferential problem created by a sparse 2 × 2 table, but a valid statistical test cannot by itself make an unrepresentative sample representative or remove confounding.

3. An association should not automatically be interpreted causally. Moore et al. (2021) caution that an observational association may be explained by confounding variables rather than a causal effect of the explanatory variable.

4. Statistical significance and substantive importance are separate issues. Meier et al. (2014) emphasize that chi-square significance does not measure the magnitude or importance of a relationship. The observed percentages and an appropriate measure of association should therefore accompany the inferential test.

5. This case does not establish a universal replacement rule. Fisher’s exact test is well supported in the approved sources for 2 × 2 contingency tables, and Lovric (2011) discusses extensions of exact procedures to larger tables. This case does not imply that Fisher’s exact test should automatically replace chi-square for every sparse table regardless of dimensions, sampling design, or research question.

The decision rule to remember

The central decision is not simply “categorical data = chi-square.”

For a two-way table, use the following sequence:

1. Start with the research question and design.
2. Construct the table of observed counts.
3. Describe the pattern with appropriate percentages.
4. Calculate expected counts under independence.
5. Assess whether the chi-square approximation is adequate.
6. Use an exact procedure such as Fisher’s exact test for a sparse 2 × 2 table when appropriate.
7. Interpret statistical evidence alongside the size and pattern of the association.

References

Devore, J. L., & Berk, K. N. (2012). Modern mathematical statistics with applications (2nd ed.). Springer.

Lovric, M. (Ed.). (2011). International encyclopedia of statistical science. Springer. https://doi.org/10.1007/978-3-642-04898-2

Meier, K. J., Brudney, J. L., & Bohte, J. (2014). Applied statistics for public and nonprofit administration (9th ed.). Cengage Learning.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2021). Introduction to the practice of statistics (10th ed.). Macmillan Learning.