A topic in the Open Knowledge Graph — a free, open map of 15,290 topics and the order to learn them in.

Factor Analysis and Dimensionality Reduction

Graduate Depth 88 in the knowledge graph ☐ I know this ☆ Set as goal
1topic build on this
461prerequisites beneath it
See this on the map →
Basis and DimensionEigenvalues and Eigenvectors+4 more→→Structural Equation Modeling with Latent Variables
factor-analysis latent-variables dimensionality communalities

Core Idea

Develops exploratory and confirmatory factor analysis for identifying latent constructs underlying observed variables. Covers factor extraction methods, rotation, interpretation of loadings, communalities, and deciding on dimensionality. Applications to instrument development and scale validation in social science.

How It's Best Learned

Conduct exploratory factor analysis on survey items, compare extraction and rotation methods, create scree plots and interpret patterns, conduct confirmatory factor analysis on independent sample.

Common Misconceptions

Explainer

From your work on eigenvalues and eigenvectors, you know that a matrix can be decomposed into directions (eigenvectors) and their associated scaling magnitudes (eigenvalues). Factor analysis applies this intuition to a correlation matrix among observed variables. Imagine you give people a 20-item survey about political attitudes — each item is a variable. Factor analysis asks: can the correlations among these 20 items be explained by a smaller number of underlying latent factors that we never directly measured? If items about economic policy all correlate with each other but not with items about social policy, that pattern suggests two underlying dimensions (factors), not twenty independent ones.

The mechanics start with the correlation matrix. Factor extraction — whether via principal axis factoring, maximum likelihood, or other methods — finds the linear combinations of observed variables that capture the most shared variance. The first factor accounts for the most common variance, the second for the next most, and so on. Each factor has an eigenvalue representing how much variance it explains. You then face a retention decision: how many factors are real signal versus noise? Common criteria include Kaiser's rule (keep factors with eigenvalue > 1), the scree plot (look for the "elbow"), and parallel analysis (compare eigenvalues to those from random data). None of these is definitive — this is a judgment call informed by theory.

Once factors are extracted, rotation is used to make them interpretable. Unrotated solutions are mathematically clean but often theoretically murky — every item loads moderately on every factor. Rotation reallocates variance so that items load strongly on one factor and weakly on others. Orthogonal rotation (like Varimax) keeps factors uncorrelated; oblique rotation (like Oblimin) allows factors to correlate, which is more realistic when underlying constructs are related. The key output is the loading matrix — a table showing how strongly each item relates to each factor. Strong loadings (above .40 or .50 by convention) define what a factor "is," and naming the factor requires reading the common thread among high-loading items.

Exploratory factor analysis (EFA) is used when you don't know the factor structure in advance — you let the data suggest it. Confirmatory factor analysis (CFA) is used when you have a theoretical model specifying which items load on which factors, and you test whether the data fit that model. These are not interchangeable: running EFA to find a structure and then immediately running CFA on the same data to "confirm" it is circular — you need an independent sample to truly test. CFA requires specifying which loadings are free (estimated) and which are fixed to zero, and it produces model fit statistics (CFI, RMSEA, SRMR) that tell you how well the hypothesized structure matches the observed correlations. Good fit means your measurement model is plausible; it does not prove the latent constructs exist or are well-measured.

What did you take from this?

Topics in reflective domains aren't scored by quiz answers. Read, reflect, and mark when you've thought it through.

Quiz me anyway →

Prerequisite Chain

Understanding Zero → The Number Zero → Counting to Five → Counting to 10 → Counting to 20 → Counting a Set of Objects Up to 20 → Cardinality: The Last Number Counted → Matching Numerals to Quantities → Subitizing Small Quantities → Addition Within 10 → Number Bonds to 10 → Addition Within 20 → Doubles and Near Doubles → Doubles Facts Within 10 → Near Doubles Facts Within 20 → Mental Math Strategies for Addition → Mental Math: Adding and Subtracting Tens → Addition Within 100 → Repeated Addition as Multiplication → Multiplication as Equal Groups → Multiplication: Arrays → Basic Multiplication Facts (0s, 1s, 2s, 5s, 10s) → Multiplication Facts Within 100 → Division as Equal Sharing → Division as Grouping (Measurement Division) → Division: Grouping (Repeated Subtraction) Model → Division: Fair Sharing Model → Division as Equal Sharing → Division as Grouping → Basic Division Facts → Division Facts Within 100 → Multiplication and Division Fact Families → Relationship Between Multiplication and Division → Division Facts as Inverse of Multiplication → Remainders and Quotients in Division → Division Word Problems → Multi-Step Word Problems → Solving Multi-Step Word Problems → Multiplication Word Problems → Division Word Problems → Introduction to Long Division → Factors and Multiples → Prime and Composite Numbers → Equivalent Fractions → Relating Fractions and Decimals → Decimal Place Value → Integers and the Number Line → Comparing and Ordering Integers → Absolute Value → Adding Integers → Subtracting Integers → Multiplying Integers → Dividing Integers → Unit Rates → Proportions → Percent Concept → Converting Between Fractions, Decimals, and Percents → Operations with Rational Numbers → Two-Step Equations → Solving Multi-Step Equations → Equations with Variables on Both Sides → Literal Equations → Slope-Intercept Form → Point-Slope Form → Writing Linear Equations → Parallel and Perpendicular Line Slopes → Graphing Linear Equations → Piecewise Functions → One-Sided Limits → Continuity Definition → Limits and Continuity in Multiple Variables → Functions of Several Variables → Continuity in Multiple Variables → Partial Derivatives: Definition and Computation → Differentiability in Multiple Variables → Differentiability in Multivariable Functions → Total Differential and Linear Approximation → Chain Rule for Multivariable Functions → Implicit Differentiation → Related Rates → Optimization Problems → Critical Points of Multivariable Functions → Critical Points and Classification of Extrema → Second Partial Test for Local Extrema (Hessian) → The Hessian Matrix and Second Derivative Test → Unconstrained Optimization: Finding Extrema → Optimization in Multiple Variables → Linear Regression for Social Science → Factor Analysis and Dimensionality Reduction

Longest path: 89 steps · 461 total prerequisite topics

Prerequisites (6)

Leads To (1)