How Many Participants Do We Need? A Power and Sample-Size Case Study Under Effect-Size Uncertainty

The planning problem

A research team is preparing a two-arm randomized study of an eight-week digital fatigue-management programme for employees experiencing persistent work-related fatigue. Participants will be allocated equally to the intervention or usual-support condition.

The primary outcome is a continuous fatigue score measured at the end of the intervention. Lower scores indicate less fatigue.

The team initially asks:

“How many participants do we need for 90% power?”

That question is incomplete. Power cannot be determined from the desired power level alone. For a two-sample comparison of means, planning requires an alternative difference that is important to detect, sample sizes or allocation, alpha, and an assumption about the outcome standard deviation (Moore et al., 2014). More generally, statistical power describes the probability of rejecting the null hypothesis when a specified alternative is true, and higher power corresponds to a lower probability of Type II error for that alternative (Moore et al., 2021).

That formulation produces a rationale that can be explained to a supervisor, ethics committee, reviewer, or research team.

1. Start with the primary research question

The synthetic primary research question is:

At eight weeks, is mean fatigue different between employees assigned to the digital fatigue-management programme and employees assigned to usual support?

This matters because sample-size planning should correspond to the statistical analysis that addresses the research question. A two-sample problem can arise directly from a randomized comparative experiment in which participants are assigned to two treatments, and the corresponding comparison concerns the difference between the two population means (Moore et al., 2014).

The planned primary analysis is therefore a two-sided independent two-sample comparison of mean eight-week fatigue scores, with equal allocation between groups.

This intentionally simple design makes the connection between question, analysis, assumptions, and sample size transparent. If the eventual study instead used a more complex longitudinal, multilevel, latent-variable, or missing-data model, the power calculation would need to represent that model rather than rely automatically on the calculation shown here. Model-specific power approaches, including Monte Carlo methods, can incorporate parameter values and other features of more complex models (Wang & Wang, 2012).

2. State the hypotheses before calculating N

Let μ_I denote the population mean fatigue score under the intervention and μ_C the corresponding mean under usual support.

The primary hypotheses are:

H₀: μ_I − μ_C = 0
H_A: μ_I − μ_C ≠ 0

Explicitly connecting the research hypothesis and its null counterpart helps make clear what statistical claim is being tested (Meier et al., 2014).

A two-sided alternative is used in this synthetic case because the study is intended to detect a difference in either direction rather than prespecifying that an effect in the unexpected direction should be excluded from the primary test.

3. Do not substitute “expected effect” for “effect that matters”

The team cannot defend a statement such as:

“We expect Cohen's d = 0.40.”

There is no sufficiently strong basis in this synthetic scenario for treating one standardized effect as the true expected value.

What the team can specify is a decision threshold:

The four-point threshold is a synthetic planning assumption, not an empirical estimate and not a universal benchmark.

This distinction matters. Statistical significance and substantive importance are different concepts, and sufficiently large samples can make very small effects statistically significant (Moore et al., 2014; Newton & Rudestam, 1999). For power planning, Moore et al. (2021) recommend specifying an alternative that matters scientifically or for decision making rather than merely choosing an arbitrary nonzero difference.

Newton and Rudestam (1999) similarly frame power and effect size in the context of the phenomenon under investigation, asking researchers to consider prior research, typical outcome values, and the smallest effect of practical significance.

The defensible input in this case is therefore not:

“The true effect will be d = 0.40.”

It is:

“We want to understand the sample size required to detect a four-point effect, while acknowledging uncertainty about how variable the outcome will be.”

4. Variability turns one meaningful difference into several plausible effect sizes

For a two-sample mean comparison, the same raw mean difference can represent different standardized effects depending on the outcome standard deviation. Power calculations for the two-sample t test therefore require both the important mean difference and an assumption about variability (Moore et al., 2014).

The synthetic team considers standard deviations between 8 and 12 points plausible for planning. These values are illustrative assumptions only; they are not presented as pilot estimates or findings from previous studies.

With a four-point meaningful difference, the corresponding standardized effects are:

This is the central uncertainty in the case. The research team does not have to claim that one of these standardized effects is “the” expected effect. It can instead show what each defensible assumption implies.

5. Make alpha and desired power explicit

The synthetic primary analysis uses a two-sided significance level of α = .05.

Alpha is part of the power calculation because changing the significance level changes the evidence required to reject the null hypothesis and therefore changes power (Moore et al., 2014; Moore et al., 2021).

The team prefers 90% power for its main planning scenario but also calculates 80% power as a sensitivity benchmark. These values are planning choices for this synthetic study, not universal standards.

6. Sensitivity analysis: what happens when the assumed effect changes?

The table below was generated computationally for this case using a two-sided, equal-allocation, independent two-sample t-test power calculation with α = .05.

All values are synthetic planning outputs.

The calculation illustrates why an unexplained effect-size assumption can dominate a sample-size justification. Under otherwise identical assumptions, moving from d = 0.50 to d = 0.30 changes the 90%-power requirement from 172 to 470 analyzable participants.

That direction is expected: effects farther from the null are easier to distinguish, whereas greater variability makes differences harder to detect; increasing sample size increases the information available to distinguish an alternative from the null (Moore et al., 2014; Moore et al., 2021).

7. Express the sensitivity analysis in the original outcome units

Because the team has defined practical relevance as a four-point difference, another useful sensitivity analysis holds that difference constant and varies the assumed standard deviation.

Synthetic sensitivity analysis for a four-point difference, α = .05, two-sided, 90% power:

This table gives the stakeholder a much better answer to “Why this sample size?” than an isolated output from power software.

The required sample size is not unstable because the software is unreliable. It changes because the underlying planning assumptions change.

8. What about attrition?

The sample size required by the statistical calculation is the number of analyzable participants required under its assumptions. If participants are expected to become unavailable for the primary outcome, recruitment may need to exceed that number.

Attrition is not only a loss of observations. An analysis restricted to study completers reduces sample size, and attrition can also introduce bias when loss is related to study condition or outcome processes (Newton & Rudestam, 1999).

For illustration, suppose the team chooses the scenario with 266 analyzable participants and wants to examine recruitment requirements under several possible loss rates.

The arithmetic recruitment target is:

required recruitment = analyzable N / expected retained proportion

All values below are synthetic.

These calculations should not be interpreted as a claim that attrition will actually equal 10%, 15%, or 20%. They show how recruitment requirements change if those planning assumptions are used.

9. Bring feasibility into the decision

Suppose the fictional project team estimates that approximately 320 participants can realistically be recruited within its time and resource constraints.

That feasibility constraint changes the discussion.

At the central synthetic assumption of a four-point meaningful difference and SD = 10 (d = 0.40), 90% power requires 266 analyzable participants. Allowing for 15% loss gives a recruitment target of approximately 313 participants, which is within the hypothetical feasibility limit.

But if variability is SD = 12, the same four-point effect corresponds to only d ≈ 0.33. Approximately 382 analyzable participants—and about 450 recruits under 15% loss—would then be required for 90% power.

10. What does the feasible sample actually buy us?

Consider several candidate analyzable sample sizes.

Synthetic achieved-power calculations for a four-point difference:

The table prevents a common planning mistake: describing a study simply as “90% powered” without saying 90% power for what effect under what variability assumption.

Power belongs to a specified alternative and set of design assumptions rather than to a sample size in isolation (Moore et al., 2021).

11. Why not choose the worst-case number automatically?

The largest sample in the sensitivity range is not automatically the correct answer.

A sample size of 382 analyzable participants would protect the 90%-power objective for the four-point effect under the synthetic SD = 12 scenario. But after allowing 15% loss, approximately 450 recruits would be required—well above the fictional project's feasible recruitment of about 320.

The decision therefore has to expose the trade-off rather than manufacture certainty.

This is consistent with the broader lesson that sample-size requirements depend on the analysis and characteristics of the planned study. Wang and Wang (2012), discussing SEM specifically, caution that no single sample-size rule applies to all situations and show how model-based and Monte Carlo approaches can be used when requirements depend on model characteristics. The same planning principle applies here: the calculation should represent the study actually being proposed.

12. When would this simple calculation stop being adequate?

This case uses an intentionally straightforward two-group comparison. The calculation would need reconsideration if the primary analysis changed materially—for example, if the research question became longitudinal or required a substantially more complex model.

For complex models, simulation can be used to specify hypothesized population parameter values, repeatedly generate data under those assumptions, fit the planned model, and examine the resulting power and precision. Wang and Wang (2012) demonstrate this model-based logic for SEM and emphasize that the necessary population values must themselves be specified from theoretical or empirical assumptions.

13. The final rationale should answer the reviewer, not just the software

The fictional research team can now write a rationale such as this:

Sample-size rationale. The primary research question concerns the difference in mean eight-week fatigue scores between two equally allocated randomized groups. The primary analysis is therefore planned as a two-sided independent comparison of means with α = .05. Rather than assuming one precisely known expected standardized effect, planning was based on a synthetic minimum practically meaningful difference of four outcome points and sensitivity to uncertainty in the outcome standard deviation. Under an SD of 10 points, the meaningful difference corresponds to d = 0.40, for which 266 analyzable participants (133 per group) provide approximately 90% power. Allowing for a synthetic 15% loss to primary-outcome analysis gives a recruitment target of approximately 313 participants. Sensitivity analysis shows that the required analyzable sample ranges from 172 when SD = 8 (d = 0.50) to 382 when SD = 12 (d ≈ 0.33) for 90% power. The planned target of 266 analyzable participants is therefore conditional on variability being near the central planning assumption; it would not provide 90% power if variability were near the upper end of the plausible range. Because recruitment above approximately 320 participants is assumed infeasible in this synthetic case, the team would document this residual risk rather than claim that the study is adequately powered under every plausible scenario.

That is more defensible than:

“G*Power said we need 266 participants.”

The first statement identifies the question, analysis, meaningful effect, variability assumption, alpha, desired power, attrition allowance, sensitivity range, feasibility constraint, and residual uncertainty.

The second provides only a number.

14. A practical checklist for defending a proposed sample size

Before presenting a sample-size calculation to a reviewer, supervisor, ethics committee, or research team, the researcher should be able to answer:

• What is the primary research question?
• Which planned statistical analysis directly answers it?
• What are the null and alternative hypotheses, where hypothesis testing is relevant?
• What effect would be practically or scientifically meaningful to detect?
• Why is that effect meaningful?
• Which variability or nuisance-parameter assumptions enter the calculation?
• What alpha level is being used?
• What power is desired, and for which specified alternative?
• Which inputs are well supported and which remain uncertain?
• How much does required N change across plausible assumptions?
• How will loss to analysis affect recruitment requirements?
• Is the resulting recruitment target feasible?
• What assumption, if wrong, would most change the recommendation?

Conclusion

The hardest part of sample-size planning is often not solving the power equation. It is deciding what belongs in it.

Power for a comparison of means depends on the alternative difference, variability, significance level, and sample size, so an exact required N necessarily reflects assumptions about those quantities (Moore et al., 2014; Moore et al., 2021). Statistical significance should also not be confused with practical importance; the effect worth detecting has to be interpreted in the substantive context of the research question (Newton & Rudestam, 1999).

For this synthetic case, 266 analyzable participants and a recruitment target of approximately 313 under 15% loss form the preferred planning scenario because they provide approximately 90% power for a four-point meaningful difference when SD = 10. They are not presented as magical numbers. If SD were 12, the same study would have only about 77% power at N = 266, and approximately 382 analyzable participants would be required for 90% power.

The defensible conclusion is therefore conditional:

That is the sample-size rationale a reviewer can evaluate.

References

Meier, K. J., Brudney, J. L., & Bohte, J. (2014). Applied statistics for public and nonprofit administration (9th ed.). Cengage Learning.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2014). Introduction to the practice of statistics (8th ed.). W. H. Freeman.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2021). Introduction to the practice of statistics (10th ed.). Macmillan Learning.

Newton, R. R., & Rudestam, K. E. (1999). Your statistical consultant: Answers to your data analysis questions. SAGE Publications.

Wang, J., & Wang, X. (2012). Structural equation modeling: Applications using Mplus. John Wiley & Sons.