PCA or Factor Analysis? The Same Variables, Two Different Research Questions

The research situation

A research team administers a 12-item workplace survey to 480 synthetic respondents. Each item uses a seven-point response scale from 1 = strongly disagree to 7 = strongly agree.

The items deliberately form correlated clusters:

The labels are deliberately withheld from the analysis. The point is to ask what the statistical procedure itself is being asked to accomplish.

The stakeholders then present two goals.

Those questions sound related. Statistically, however, they are not the same.

The decision comes before the software

Tabachnick and Fidell (2013) make the distinction in terms of the goals and variance being analyzed. PCA analyzes the variance of the observed variables and seeks components that successively account for as much of that variance as possible. Common factor analysis focuses on the variance shared among variables and attempts to represent their correlations through underlying common factors.

Lovric (2011) similarly describes PCA as a dimension-reduction technique that forms linear combinations of the observed variables, whereas factor analysis belongs to the latent-variable framework in which unobservable variables are used to explain correlations among observed variables.

The decision can therefore be summarized as follows:

Goal A: Reduce 12 observed variables to a smaller set of composites

For Stakeholder A, the problem is operational: reduce dimensionality.

That objective points naturally toward PCA. PCA finds linear combinations of the observed variables called principal components. The first component is selected to have the largest possible variance; subsequent components successively maximize remaining variance while being uncorrelated with earlier components in the unrotated solution (Lovric, 2011; Tabachnick & Fidell, 2013).

What variance does PCA use?

With standardized variables, PCA begins with ones on the diagonal of the correlation matrix. In this formulation, each observed variable contributes its full unit variance. Common, unique, and error-related variation are therefore not separated before components are extracted (Tabachnick & Fidell, 2013).

This feature makes sense for Stakeholder A. The objective is not to construct a model of latent causes. It is to preserve as much information in the observed variables as possible while using fewer dimensions.

Synthetic PCA results

PCA of the 12 standardized synthetic variables produces the following first six eigenvalues:

The first three components summarize approximately 63.7% of the total standardized-variable variance. The research team could therefore replace 12 observed variables with three component scores if that degree of compression is adequate for its intended use.

PCA retention should be treated as a substantive dimensionality-reduction decision rather than as a universal percentage rule. Lovric (2011) notes that decisions about how many components to retain depend on context and that several approaches are available.

What is a principal component?

A component is a weighted linear combination of the observed variables. Conceptually:

PC₁ = a₁Q₁ + a₂Q₂ + ⋯ + a₁₂Q₁₂

The coefficients are chosen so that the first component captures maximum variance, with subsequent components accounting for additional variance subject to the PCA constraints (Lovric, 2011; Tabachnick & Fidell, 2013).

That direction matters.

Rotation can make the PCA easier to interpret

An unrotated PCA is optimized for variance extraction, not necessarily for substantive simplicity. Rotation can be applied after retaining components to make their relationships with the observed variables easier to interpret. Rotation changes the orientation used to describe the retained multidimensional space rather than turning components into latent factors (Lovric, 2011; Tabachnick & Fidell, 2013).

For illustration, an orthogonal varimax rotation of the first three synthetic components produces a simple pattern. After reordering the rotated components for presentation, the approximate loadings are:

The clusters are substantively recognizable. Component 1 is dominated by Q1–Q4, Component 2 by Q5–Q8, and Component 3 by Q9–Q12.

The stakeholder could give those components descriptive labels such as work discretion, coworker support, and work depletion.

But the wording of the conclusion matters.

“The 12 observed survey variables can be represented compactly by three interpretable weighted composites in this synthetic dataset.”

Goal B: Investigate underlying latent constructs

Stakeholder B has a different objective. The team is not merely trying to compress a spreadsheet. It proposes that correlations among the observed responses arise because respondents differ on a smaller number of unobserved constructs.

That objective points toward common factor analysis.

Factor analysis belongs to a latent-variable framework. A latent variable cannot be observed directly; observed variables serve as manifestations or indicators through which the underlying dimension is investigated (Lovric, 2011). Wang and Wang (2012) likewise describe latent factors as unobservable variables that must be estimated indirectly through observed indicators.

Shared variance is now central

The distinction becomes especially clear on the diagonal of the correlation matrix.

PCA analyzes the full standardized variance of each variable. Common factor analysis instead estimates the amount of variance each observed variable shares with the other observed variables—the variable's communality—and analyzes that common variance. Unique and error variance are not treated as common-factor variance (Tabachnick & Fidell, 2013).

Suppose Q1 has substantial variation that is specific to its wording or measurement. PCA still incorporates that observed variance into its component solution. A common-factor model attempts instead to isolate the part of Q1 that participates in the common covariance structure.

That is why changing the stakeholder's question changes the appropriate model even though Q1 through Q12 have not changed.

The extraction logic is different

Several common-factor extraction approaches exist, and they do not all optimize the same criterion. For example, principal-factor extraction begins with estimates of communalities rather than ones on the diagonal and seeks factors from common variance, whereas maximum-likelihood factor extraction estimates loadings under a likelihood framework. Other approaches seek to minimize discrepancies between observed and reproduced correlations (Tabachnick & Fidell, 2013).

For the synthetic illustration, a three-factor common-factor solution was estimated from the same 12 variables.

The estimated communalities were approximately:

These numbers illustrate the key distinction. The factor model is not treating every standardized variable as contributing a full variance of 1 to the common-factor solution. Instead, only part of each variable's variance is represented as common.

What is a latent factor?

The conceptual direction now reverses.

The observed variables are modeled as indicators of underlying factors. Wang and Wang (2012) describe factor loadings as links between latent factors and their observed indicators; the observed measure also contains a residual or measurement-error component.

A simplified representation is:

Q_i = λ_i1F₁ + λ_i2F₂ + λ_i3F₃ + e_i

where the F's are latent factors, the λ's are factor loadings, and e_i represents variance not accounted for by the common factors.

The research interpretation is therefore different from PCA. The question is whether a relatively small number of underlying dimensions can account for the observed correlation pattern.

Rotation is about interpretability, not changing the model into something else

Unrotated factor solutions are often difficult to interpret. Rotation is therefore commonly used to obtain a clearer structure. Tabachnick and Fidell (2013) emphasize that rotation is intended to improve interpretability and scientific usefulness rather than improve the mathematical fit of an orthogonally rotated solution.

A major decision is whether the factors should be constrained to remain uncorrelated.

If the underlying processes are theoretically expected to relate to one another, an oblique solution can be more appropriate than forcing independence merely for convenience (Tabachnick & Fidell, 2013).

For this survey, it is plausible that perceived discretion, coworker support, and depletion could be related. A researcher investigating latent constructs should therefore consider an oblique factor solution rather than automatically selecting varimax simply because it appears as a familiar software option.

When oblique rotation is used, interpretation also requires attention to the distinction between the pattern and structure matrices. The pattern matrix reflects the distinctive relationship of a factor with a variable after overlap among correlated factors is accounted for, whereas the structure matrix contains correlations between variables and factors that also incorporate factor overlap. Tabachnick and Fidell (2013) note that researchers commonly emphasize the pattern matrix while considering the factor correlations and other output.

Why the two analyses may look surprisingly similar

At first glance, the synthetic PCA and common-factor results both suggest three clusters:

• Q1–Q4;
• Q5–Q8;
• Q9–Q12.

This resemblance is precisely where researchers can get into trouble.

Similar numerical patterns do not make PCA and factor analysis equivalent. Tabachnick and Fidell (2013) show that extraction methods can yield similar-looking loading patterns when the data have a strong structure even though the statistical models and variance being analyzed differ.

The distinction is therefore conceptual as well as computational.

Those questions can sometimes lead to visually similar loading tables. They still support different claims.

Why software menus make the distinction easy to miss

Statistical software contributes to the confusion.

PCA and common factor analysis may be offered within the same “factor analysis” procedure. Users may select the same variables, request an extraction, choose a number of dimensions, apply a rotation, and receive matrices of loadings that look almost identical in layout.

Lovric (2011) explicitly notes that confusion arises partly because widely used software packages may treat PCA as though it were a special case of factor analysis, even though PCA is not factor analysis.

Tabachnick and Fidell (2013) likewise discuss PCA and several factor-extraction methods within the same practical workflow and note that both IBM SPSS and SAS provide these procedures. The common interface is a software convenience; it does not erase the methodological distinction.

A menu sequence such as:

Analyze → Dimension Reduction → Factor → Extraction

can therefore hide the most important decision.

The researcher must still decide what is being modeled.

Same variables, different claims

Stakeholder A: defensible PCA interpretation

The team can say:

“The 12 survey variables contained substantial redundancy. Three principal components provided a lower-dimensional representation of the observed variables and accounted for approximately 63.7% of their total standardized variance in this synthetic dataset.”

The team can then use the retained component scores as compact observed-variable summaries when that serves the subsequent research purpose.

PCA is particularly well aligned with this goal because dimensionality reduction is one of its central uses (Lovric, 2011; Tabachnick & Fidell, 2013).

What Stakeholder A cannot claim from PCA alone

Stakeholder B: defensible factor-analysis interpretation

The second team can instead say:

“A three-factor common-factor model provided an interpretable representation of the shared covariance among the 12 synthetic survey items, with Q1–Q4, Q5–Q8, and Q9–Q12 primarily associated with different underlying dimensions.”

That interpretation is explicitly about common latent dimensions rather than merely compressing observed variables.

Factor analysis is designed for this type of latent-variable question: latent variables are introduced to account for associations among manifest variables (Lovric, 2011).

What Stakeholder B still cannot claim

A practical decision framework

When PCA and factor analysis both appear plausible, begin with the research objective rather than the software menu.

This is not a contest in which one technique is universally better. Each method answers a different question.

The central lesson from the synthetic case

Nothing about the dataset changed.

There were still 480 synthetic respondents. There were still 12 correlated survey variables. The correlation matrix was the same.

What changed was the research question.

That distinction is more important than whether two loading matrices happen to look similar.

References

Lovric, M. (Ed.). (2011). International encyclopedia of statistical science. Springer. https://doi.org/10.1007/978-3-642-04898-2

Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Pearson.

Wang, J., & Wang, X. (2012). Structural equation modeling: Applications using Mplus. John Wiley & Sons.