ANOVA or ANCOVA? Handling Baseline Differences in a Group-Comparison Study

The Research Problem

Suppose a researcher evaluates two versions of a six-week academic-skills intervention against a control condition. The outcome is a continuous follow-up performance score, with higher scores indicating better performance.

Participants are assigned to three groups:

• Control: n = 60
• Intervention A: n = 60
• Intervention B: n = 60

Before the intervention begins, every participant completes the same performance measure. This baseline performance score is therefore available before treatment and is relevant to later performance.

The researcher initially proposes:

“We have three independent groups and a continuous outcome, so we will run a one-way ANOVA on the follow-up scores.”

That analysis is legitimate for one question. But it is not the only question the study can ask.

What Would the One-Way ANOVA Test?

One-way ANOVA is designed to compare the means of a quantitative response across two or more populations or groups defined by one factor. Its omnibus null hypothesis is that all population group means are equal; the alternative is that at least some group means differ (Devore & Berk, 2012; Moore et al., 2014, 2021).

Suppose the synthetic descriptive results are:

The unadjusted follow-up difference between Intervention B and Control is therefore 9 points.

The appropriate conclusion is that the observed follow-up means are not all equal. The omnibus ANOVA does not, by itself, identify which particular means differ; specific contrasts or a suitable multiple-comparison procedure are needed for those questions (Moore et al., 2014, 2021).

More importantly, this analysis has not used the baseline scores at all.

Why Baseline Changes the Analytical Question

Baseline performance is relevant because participants who start with higher performance may also tend to have higher follow-up performance. A covariate is useful in ANCOVA when it is associated with the dependent variable; otherwise there is little reason to include it for covariance adjustment (Tabachnick & Fidell, 2013).

Here the baseline means are noticeably different:

• Control: 66
• Intervention A: 70
• Intervention B: 74

Intervention B therefore begins approximately eight points above Control.

If the research question instead concerns follow-up differences after accounting for baseline performance, the model needs to incorporate baseline.

What Would ANCOVA Ask?

A one-way ANCOVA assesses differences between groups on a dependent variable after statistically accounting for one or more covariates. Tabachnick and Fidell (2013) describe one-way ANCOVA specifically as assessing group differences on a single dependent variable after covariate effects have been statistically removed.

A simple model can be represented as:

Follow-up performance = group + baseline performance + error

This is a different estimand from the unadjusted ANOVA comparison.

ANCOVA can also reduce residual variation when a covariate predicts the dependent variable, thereby increasing the sensitivity of tests of group effects. Another purpose is to produce group comparisons adjusted for the covariate (Tabachnick & Fidell, 2013).

Unadjusted and Adjusted Results Can Tell Different Stories

Suppose baseline performance is positively related to follow-up performance and the fitted synthetic ANCOVA estimates a common baseline slope of approximately 0.65 points of follow-up performance per baseline point.

At a common baseline value, the estimated adjusted means are:

The conclusion has changed quantitatively. The groups still show evidence of differences after adjustment, but the estimated separation between them is substantially smaller.

What Exactly Does the Baseline Covariate Do?

Conceptually, ANCOVA combines features of ANOVA and regression. Analysis involving categorical and quantitative predictors can be expressed through the general linear-model framework, with the quantitative predictor serving as a covariate (Devore & Berk, 2012).

In this case, baseline performance helps account for predictable variation in follow-up performance. ANCOVA then assesses whether group membership contributes to differences in the outcome after that relationship has been incorporated into the model (Tabachnick & Fidell, 2013).

A baseline measure of the same outcome is a classic application of covariance analysis in pretest–posttest settings (Tabachnick & Fidell, 2013).

But “adjusted” should not be interpreted as “design problems removed.”

ANCOVA Does Not Repair a Weak Study Design

Statistical adjustment and research design solve different problems.

Tabachnick and Fidell (2013) explicitly caution that neither ANOVA nor ANCOVA can establish that changes in a dependent variable were caused by an independent variable. Causal interpretation depends on design features such as how participants were assigned, manipulation of the independent variable, and experimental controls.

Research design is therefore still central. Random assignment, control of extraneous variables, and protection against confounds are design issues rather than benefits automatically supplied by a later statistical model (Adams & Lawrence, 2018).

For example, imagine that participants in this synthetic study actually chose their own intervention. Intervention B attracts participants who already have greater motivation, more study time, and higher baseline performance.

Adding baseline performance to an ANCOVA may adjust for the measured baseline variable. It does not demonstrate that the groups are otherwise exchangeable, nor does it automatically remove bias associated with other uncontrolled group differences.

Assumptions and Checks Before Interpreting ANCOVA

ANCOVA inherits familiar linear-model requirements while adding conditions related specifically to covariate adjustment. Tabachnick and Fidell (2013) identify normality, homogeneity of variance, linear relationships between covariates and the dependent variable, reliability of covariates, and homogeneity of regression as relevant considerations.

1. Independence and study structure

Observations must have an appropriate independence structure for the conventional model. For ANOVA, the standard model treats the deviations as independent, and the design or sampling process is fundamental to determining whether the observations can reasonably be treated that way (Moore et al., 2021).

Repeated observations, clustering, or other dependencies should therefore not simply be ignored in order to fit an ordinary one-way model.

2. Distributional behavior and residuals

The one-way ANOVA model assumes independent, normally distributed deviations with a common standard deviation across groups. In practice, diagnostics should consider distributions, unusual observations, group variability, and residual behavior rather than treating a single assumption test as the whole diagnostic process (Moore et al., 2021).

3. Baseline–outcome linearity

ANCOVA assumes an appropriate relationship between the continuous covariate and the dependent variable. A clearly nonlinear relationship may require reconsideration of the model rather than automatic use of a simple linear baseline adjustment (Tabachnick & Fidell, 2013).

4. Reliability of the covariate

ANCOVA conventionally assumes that covariates are measured without error. Tabachnick and Fidell (2013) note that this assumption becomes less plausible for variables measured with imperfect psychometric reliability.

A baseline measure should therefore not be treated as error-free merely because it was collected before the intervention.

5. Homogeneity of regression slopes

This is particularly important.

The conventional ANCOVA adjustment uses a common within-group regression relationship between the dependent variable and covariate. Homogeneity of regression means that this relationship is assumed to be the same across the groups (Tabachnick & Fidell, 2013).

What If Baseline Predicts the Outcome Differently Across Groups?

Suppose the researcher fits:

Follow-up = group + baseline + group × baseline

and obtains a synthetic interaction test:

F(2, 174) = 0.74, p = .48.

There is little evidence in these synthetic data that the baseline–follow-up relationship differs across groups. A common-slope ANCOVA is therefore more defensible.

But imagine instead that the interaction were substantial.

The interpretation would change. Heterogeneity of regression means that the relationship between the covariate and outcome varies across levels of the group variable. In that situation, a single adjusted group difference can be misleading because the amount or direction of the group difference may depend on the covariate level (Tabachnick & Fidell, 2013).

The research question may then become:

That question requires interpretation of the interaction rather than forcing a single common-slope adjusted comparison.

Planned Contrasts Versus Post-Hoc Comparisons

A significant omnibus ANOVA or ANCOVA does not answer every pairwise question.

For one-way ANOVA, Moore et al. (2021) distinguish contrasts, which express specific comparisons among population means, from procedures designed to conduct multiple comparisons. Contrasts are particularly appropriate for focused questions specified before examining the results (Moore et al., 2021).

For this intervention, two scientifically motivated comparisons could have been specified in advance:

1. Any intervention versus Control: average of Intervention A and B compared with Control.
2. Intervention B versus Intervention A: whether the enhanced intervention differs from the standard intervention.

These comparisons map directly onto the intervention design and can also be framed for adjusted group means in an ANCOVA setting (Tabachnick & Fidell, 2013).

Suppose the synthetic adjusted comparisons are:

The first comparison suggests higher adjusted follow-up performance for the intervention conditions collectively. The second does not provide clear evidence that Intervention B outperforms Intervention A.

If the researcher instead examines many pairwise comparisons after seeing the data, multiplicity becomes relevant. Multiple-comparison procedures are designed to address inference across a collection of comparisons rather than treating every pairwise test as an isolated analysis (Devore & Berk, 2012; Moore et al., 2014, 2021).

Effect Interpretation: Do Not Stop at p

The inferential result should not be reduced to whether p is below .05.

For the unadjusted ANOVA, η² = .15 describes the proportion of outcome variation associated with the group structure in that fitted analysis. For the synthetic ANCOVA, partial η² = .04 describes the group contribution within the adjusted model.

More directly interpretable for this case are the estimated mean differences and their uncertainty:

• Unadjusted Intervention B–Control difference: 9.0 points
• Baseline-adjusted Intervention B–Control difference: 3.8 points
• Adjusted Intervention B–Intervention A difference: 1.4 points

Statistical significance alone does not establish practical importance. Researchers should interpret the size and uncertainty of the estimated difference in relation to the substantive measurement scale and research question (Sekaran & Bougie, 2016).

In this synthetic case, no externally validated threshold for a “meaningful” number of performance points has been supplied. The analysis therefore should not invent one.

Why the ANOVA and ANCOVA Results Differ

The difference is understandable from the descriptive data.

Intervention B begins with the highest baseline mean and finishes with the highest follow-up mean. Because baseline performance predicts follow-up performance, some of Intervention B's unadjusted follow-up advantage is associated with where its participants started.

ANCOVA asks what the group comparison looks like after accounting for that baseline relationship.

The adjustment changes the comparison because baseline is both relevant to the outcome and distributed differently across the groups.

Common Reporting Mistakes

A Practical Decision Framework

When deciding between ANOVA and ANCOVA in a group-comparison study with a baseline measure, work through the decision in this order:

1. Define the comparison.
2. Examine the study design and baseline variable.
3. Choose the model.
4. Check its assumptions.
5. Interpret the adjusted or unadjusted result.

The key decision is therefore not “Which test is more sophisticated?”

How This Synthetic Study Would Be Reported

A concise results section could read:

Follow-up performance differed across the three groups in the unadjusted analysis, F(2, 177) = 15.8, p < .001, η² = .15. Mean follow-up scores were 70.0 for Control, 75.0 for Intervention A, and 79.0 for Intervention B. Because baseline performance was relevant to follow-up performance and differed descriptively across groups, a prespecified ANCOVA was used to estimate group differences conditional on baseline performance. The group effect remained statistically detectable after adjustment, F(2, 176) = 4.1, p = .018, partial η² = .04. Adjusted means were 72.6, 75.0, and 76.4 for Control, Intervention A, and Intervention B, respectively. The group × baseline interaction was not statistically significant, F(2, 174) = 0.74, p = .48. The planned comparison of the two intervention groups collectively against Control yielded an adjusted difference of 3.1 points, 95% CI [0.7, 5.5], whereas Intervention B did not clearly differ from Intervention A, adjusted difference = 1.4 points, 95% CI [−1.4, 4.2]. These adjusted comparisons should not be interpreted as evidence that statistical adjustment eliminated all possible group differences arising from the study design.

Conclusion

ANOVA and ANCOVA are not competing buttons for the same question.

A one-way ANOVA asks whether the groups differ on the outcome as observed, without baseline adjustment. ANCOVA asks whether they differ after accounting for a specified covariate, such as a relevant pre-intervention measure (Moore et al., 2021; Tabachnick & Fidell, 2013).

In this synthetic study, the unadjusted difference between Intervention B and Control was 9 points, but the baseline-adjusted difference was only 3.8 points. That discrepancy is analytically informative: the groups began at different baseline levels, and baseline was related to the follow-up outcome.

ANCOVA can make that relationship explicit. It cannot retroactively randomize participants, remove unmeasured confounding, guarantee causal identification, or otherwise transform a weak design into a strong one. Design remains the foundation of interpretation (Adams & Lawrence, 2018; Tabachnick & Fidell, 2013).

References

Adams, K. A., & Lawrence, E. K. (2018). Research methods, statistics, and applications (2nd ed.). SAGE Publications.

Devore, J. L., & Berk, K. N. (2012). Modern mathematical statistics with applications (2nd ed.). Springer.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2014). Introduction to the practice of statistics (8th ed.). W. H. Freeman.

Moore, D. S., McCabe, G. P., & Craig, B. A. (2021). Introduction to the practice of statistics (10th ed.). Macmillan Learning.

Sekaran, U., & Bougie, R. (2016). Research methods for business: A skill-building approach (7th ed.). Wiley.

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson.