Skip to content

Actuarial Statistics · Exploratory data analysis

Principal Components Analysis: Loadings, Variance Explained and Scree Plot

Updated 11 October 2026 · Fact-checked

Principal components analysis (PCA) replaces correlated variables with a few uncorrelated linear combinations that capture as much variance as possible. Find the eigenvalues, divide each by their total to get the proportion of variance explained, read the loadings to interpret each component, and use the scree plot to decide how many to keep.

Understand Principal Components Analysis

Suppose you have many numerical variables that are correlated, such as several rating factors for a motor policy. Much of the information is repeated. Principal components analysis (PCA) builds new variables, called principal components, that are linear combinations of the original ones. They are uncorrelated with each other and are ordered by how much variance they carry.

The first component (PC1) is the combination with the largest possible variance. PC2 has the largest variance among combinations uncorrelated with PC1, and so on. If p variables go in, p components come out. The gain comes from keeping only the first few, which reduces dimension with little loss of information.

The weights in each combination are called loadings. They are scaled so that the squared loadings of one component add up to 1. Large loadings (in absolute value) show which original variables drive that component. The loadings of a component are an eigenvector of the covariance or correlation matrix, and the matching eigenvalue is the variance of that component.

Choose between the covariance matrix and the correlation matrix first. Use the covariance matrix only if the variables share the same units and similar scales. Otherwise, standardise the variables (equivalent to using the correlation matrix). If you do not, the variable with the biggest scale will dominate PC1 for no real reason.

A scree plot graphs the eigenvalues (or proportion of variance) against the component number. You look for the "elbow", where the curve flattens. Components before the elbow are worth keeping. Other common rules are to keep enough components to reach a target cumulative variance, or, for a correlation matrix, to keep components with eigenvalue above 1 (the average eigenvalue).

Key rules to remember

Principal component
Z₁ = φ₁₁X₁ + φ₂₁X₂ + … + φₚ₁Xₚ
Variables are usually centred (mean subtracted) first. The score of an observation is this sum using its centred values. Standardise too if you use the correlation matrix.
Loading normalisation
Σⱼ φⱼᵢ² = 1 for each component i
Loading vectors of different components are also orthogonal: Σⱼ φⱼᵢ φⱼₖ = 0 for i ≠ k.
Variance of a component
Var(Zᵢ) = λᵢ
λᵢ is the i-th largest eigenvalue of the covariance (or correlation) matrix, with λ₁ ≥ λ₂ ≥ … ≥ λₚ ≥ 0.
Total variance
Σ λᵢ = trace of the matrix
For a covariance matrix this is the sum of the variances. For a correlation matrix it equals p, the number of variables.
Proportion of variance explained
PVEᵢ = λᵢ ÷ Σ λⱼ
Cumulative PVE for the first k components is (λ₁ + … + λₖ) ÷ Σ λⱼ.
Eigenvalue equation
Σφ = λφ, i.e. (Σ − λI)φ = 0
Solve det(Σ − λI) = 0 for λ, then find φ and scale it to length 1. Σ here is the covariance or correlation matrix.
Correlation of component with variable
Corr(Zᵢ, Xⱼ) = φⱼᵢ √λᵢ ÷ sⱼ
sⱼ is the standard deviation of Xⱼ. For a correlation matrix sⱼ = 1, so it is simply φⱼᵢ √λᵢ.

How to solve Principal Components Analysis questions

Use this order for any PCA question, whether you are given the matrix, the eigenvalues or R output.

  1. 1State the matrix used: covariance if variables are on the same scale, correlation if not. Say so in your answer.
  2. 2Get the eigenvalues. For a 2×2 matrix, solve det(Σ − λI) = 0. Check that they add up to the trace (or to p for a correlation matrix).
  3. 3Compute each proportion of variance explained as λᵢ ÷ Σλ, then the cumulative proportions.
  4. 4Decide how many components to keep, using the scree plot elbow, a cumulative variance target, or eigenvalue above the average. Name the rule you use.
  5. 5Find the loadings by solving (Σ − λI)φ = 0, then scale so that the squared loadings sum to 1. Check the sign and orthogonality.
  6. 6Interpret each kept component by looking at the largest absolute loadings and their signs. Give it a plain meaning, such as an overall size or a contrast between two groups.
  7. 7If asked for scores, centre (and scale) the observation, then multiply by the loadings and add.
  8. 8State any limits: PCA is linear, depends on scaling, and components may be hard to interpret.

Quickest way: Eigenvalue table shortcut

When to use it: Use when you are given eigenvalues or R output (such as summary of prcomp) and must find variance explained or how many components to keep.

  1. Add the eigenvalues once. Check against the trace or p.
  2. Divide each eigenvalue by the total and write the running total in a small table.
  3. Read off the first row that reaches the target cumulative proportion.
  4. For a correlation matrix, also mark eigenvalues above 1 as a cross-check.
  5. For loadings, pick the largest absolute values per component and describe the pattern in one sentence. Do not compute anything not asked for.

Common mistakes in Principal Components Analysis

  • Running PCA on the covariance matrix when variables have very different units or scales.

    Students forget that PCA is not scale invariant, so the largest-variance variable swamps PC1.

    Fix: Standardise the variables, or use the correlation matrix, unless all variables share the same units and scale. State your choice.

  • Dividing by the wrong total when finding the proportion of variance explained.

    Students divide by p for a covariance matrix, or by the largest eigenvalue.

    Fix: Always divide by the sum of all eigenvalues. That sum is the trace, and it equals p only for a correlation matrix.

  • Forgetting to scale the loading vector to length 1.

    Solving (Σ − λI)φ = 0 gives a direction, such as (2, 1), and students stop there.

    Fix: Divide by the square root of the sum of squares, here √5. Then check that the squared loadings sum to 1.

  • Reading the sign of a loading as meaning something absolute.

    Students think a negative PC1 loading means a negative effect.

    Fix: Flipping all signs in one component gives an equally valid component. Only the relative signs and sizes matter for interpretation.

  • Treating the scree plot elbow as an exact rule.

    Students want one correct answer, but the elbow is a judgement.

    Fix: Say where the curve flattens, then support it with the cumulative variance or the eigenvalue above 1 rule. Give a reasoned number.

  • Saying PCA components are the same as the original variables or that they are always interpretable.

    Students mix up dimension reduction with variable selection.

    Fix: Remember each component mixes all the original variables. Dimension is reduced, but you still need all the original variables to compute the scores.

Worked examples

Example 1

A PCA is run on the correlation matrix of four variables. The eigenvalues are 2.4, 1.1, 0.3 and 0.2. (a) Find the proportion of variance explained by each component. (b) How many components are needed to explain at least 90% of the total variance? (c) How many components would the eigenvalue-above-1 rule keep?

Show the solution
  1. Total variance = 2.4 + 1.1 + 0.3 + 0.2 = 4.0, which equals p = 4, as expected for a correlation matrix.
  2. PVE for PC1 = 2.4 ÷ 4 = 0.60. PC2 = 1.1 ÷ 4 = 0.275. PC3 = 0.3 ÷ 4 = 0.075. PC4 = 0.2 ÷ 4 = 0.05.
  3. Cumulative: PC1 = 0.60, PC1–PC2 = 0.875, PC1–PC3 = 0.95, PC1–PC4 = 1.00.
  4. The cumulative proportion first reaches 90% at three components, since 0.875 is below 0.90 and 0.95 is above it.
  5. Eigenvalues above 1 are 2.4 and 1.1 only.

Answer: (a) 60%, 27.5%, 7.5% and 5%. (b) Three components (cumulative 95%). (c) Two components under the eigenvalue-above-1 rule. The two rules disagree, so state which one you apply and why.

Example 2

The covariance matrix of two variables X₁ and X₂ is Σ = [[5, 2], [2, 2]]. (a) Find the eigenvalues. (b) Find the normalised loadings of PC1. (c) Find the proportion of variance explained by PC1. (d) An observation has centred values (3, 1). Find its PC1 score.

Show the solution
  1. Characteristic equation: det(Σ − λI) = (5 − λ)(2 − λ) − 4 = λ² − 7λ + 6 = 0.
  2. So (λ − 6)(λ − 1) = 0, giving λ₁ = 6 and λ₂ = 1. Check: the sum is 7, which equals the trace 5 + 2.
  3. For λ = 6: (5 − 6)x + 2y = 0, so −x + 2y = 0 and x = 2y. A direction is (2, 1).
  4. Length = √(4 + 1) = √5, so φ₁ = (2 ÷ √5, 1 ÷ √5) = (0.894, 0.447). Check: 0.8 + 0.2 = 1.
  5. PVE for PC1 = 6 ÷ 7 = 0.857.
  6. PC1 score = (2 × 3 + 1 × 1) ÷ √5 = 7 ÷ 2.236 = 3.13.

Answer: (a) λ₁ = 6 and λ₂ = 1. (b) Loadings (2/√5, 1/√5) ≈ (0.894, 0.447), so PC1 is a weighted total of both variables with more weight on X₁. (c) 6/7 ≈ 85.7%. (d) Score ≈ 3.13.

Exam tips

  • Show the eigenvalue total check (trace, or p for a correlation matrix). It earns marks and catches arithmetic slips.
  • When asked to interpret loadings, name the variables with the largest absolute loadings and describe the pattern in words, for example an overall size component or a contrast.
  • For scree plot questions, give the elbow location, then back it with cumulative variance. Always say what you are trading off: fewer components against lost information.
  • In the computer-based paper, know how to run prcomp in R with scale = TRUE and read the standard deviations. Remember the standard deviations are square roots of the eigenvalues, so square them before finding proportions.
  • State the scaling choice and the assumption that PCA is a linear method every time you present results.

Practice questions from Exploratory data analysis

Principal Components Analysis in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Principal Components Analysis: frequently asked questions

What is the difference between loadings and scores in PCA?

Loadings are the weights that define each component, and they are the same for every observation. Scores are the values of the components for each observation, found by applying the loadings to its centred (and scaled) data.

How do I calculate the proportion of variance explained by principal components?

Divide each eigenvalue by the sum of all eigenvalues. For example, with eigenvalues 6 and 1, PC1 explains 6 ÷ 7, about 85.7%. Add proportions to get the cumulative variance explained.

How do I read a scree plot?

Look for the elbow, where the line stops dropping steeply and becomes flat. Keep the components before the elbow. Support your choice with the cumulative variance explained, because the elbow is a judgement.

Should I use the covariance matrix or the correlation matrix?

Use the covariance matrix only when the variables are on the same units and similar scales. Otherwise use the correlation matrix, which is the same as standardising each variable first, so no variable dominates because of its scale.

Are principal components always uncorrelated?

Yes, the components from PCA are uncorrelated with each other by construction, because the loading vectors are orthogonal. This does not mean they are independent unless the data are jointly normal.