Fundamentals of Business Mathematics and Statistics · Correlation and Regression

Probable Error and Coefficient of Determination Explained

Updated 10 October 2026 · Fact-checked

Probable error (PE) tests how reliable a correlation coefficient r is: PE = 0.6745 × (1 − r²) ÷ √N. The coefficient of determination is r², the proportion of variation in one variable explained by the other. To solve questions, compute r², then plug into the formula or read r² as a percentage.

Understand Probable Error and Coefficient of Determination

When you calculate r from a sample of N pairs, it is only an estimate. A different sample would give a slightly different r. Probable error is a measure of how much r may vary because of sampling. It helps you judge whether r can be trusted.

The rules of thumb used in the syllabus are simple. If r is less than 6 times PE, there is no real evidence of correlation. If r is more than 6 times PE, correlation is significant. Also, the limits r ± PE give a range in which the population correlation is likely to lie. These rules assume the data come from a roughly normal population and that the sample is reasonably large. Treat them as guidelines, not proof.

The coefficient of determination is r². It tells you what share of the variation in Y is explained by X (or the other way round). If r = 0.8, then r² = 0.64. This means 64% of the variation is explained by the linear relationship. The remaining 36% is unexplained and is due to other factors. That leftover share is 1 − r², called the coefficient of non-determination.

Notice that r² is never negative, so it hides the direction of the relationship. r = 0.8 and r = −0.8 both give r² = 0.64. Also, r² is smaller than |r| unless r is 0 or ±1. So r = 0.5 explains only 25% of the variation, not 50%.

Key formulas to remember

Probable error of r: PE = 0.6745 × (1 − r²) ÷ √N; N is the number of pairs of observations. Use r², not r.
Standard error of r: SE = (1 − r²) ÷ √N; PE = 0.6745 × SE.
Limits for population correlation: r − PE to r + PE; Range in which the population correlation is likely to lie.
Significance rule of thumb: If r > 6 × PE, r is significant. If r < 6 × PE, r is not significant.; Also, if r is not significant it gives no evidence of correlation. Use |r| for negative values.
Coefficient of determination: r² = (explained variation) ÷ (total variation); Usually quoted as a percentage, r² × 100.
Coefficient of non-determination: 1 − r²; Share of variation left unexplained.

How to solve Probable Error and Coefficient of Determination questions

Use this method for any question on probable error or r².

1Note what is given: r, N, or r² (or the percentage explained).
2If only r² is given, find r = √r². Take the sign from the context (positive or negative correlation).
3For probable error, find 1 − r² first.
4Compute √N, then PE = 0.6745 × (1 − r²) ÷ √N.
5For limits, calculate r − PE and r + PE.
6For significance, compare |r| with 6 × PE.
7For r², multiply by 100 to get the percentage explained. Use 1 − r² for unexplained variation.
8Check that your answer matches one option and the units (percentage or decimal) match.

Quickest way: Pick N so that √N is clean

When to use it: For PE numericals in MCQs, where N is usually a perfect square like 25, 36, 49 or 64.

Find 1 − r² mentally using the clean value of r (0.6 gives 0.64 as r², so 1 − r² = 0.36).
Divide by √N first, then multiply by 0.6745 at the end.
Approximate 0.6745 as about two-thirds to eliminate options far away.
For r² questions, skip PE entirely: square r and convert to a percentage.
Remember r = 0.9 gives 81%, r = 0.5 gives 25%, r = 0.6 gives 36%.

Common mistakes in Probable Error and Coefficient of Determination

Using r instead of r² inside the PE formula
The formula looks like it needs r, so students write 1 − r.
Fix: Always write (1 − r²). Square r before subtracting from 1.
Treating r as the percentage of variation explained
Students confuse r with r².
Fix: Square r first. r = 0.7 means 49% explained, not 70%.
Forgetting the square root of N
Students divide by N directly.
Fix: Write √N on its own line and evaluate it before dividing.
Dropping the negative sign of r when testing significance
A negative r looks smaller than 6 × PE.
Fix: Compare the absolute value |r| with 6 × PE.
Saying r² shows the direction of correlation
r² is always positive, but students still read it as positive correlation.
Fix: Take the direction only from the sign of r.

Worked examples

Example 1

For 25 pairs of observations, the correlation coefficient is 0.8. Find the probable error and state the limits of the population correlation.

Show the solution

r = 0.8, so r² = 0.64 and 1 − r² = 0.36.
N = 25, so √N = 5.
PE = 0.6745 × 0.36 ÷ 5 = 0.6745 × 0.072 = 0.048564, approximately 0.0486.
Lower limit = 0.8 − 0.0486 = 0.7514.
Upper limit = 0.8 + 0.0486 = 0.8486.

Answer: PE ≈ 0.0486; the population correlation lies between about 0.7514 and 0.8486.

Example 2

The correlation between advertising spend and sales is 0.6 for 36 observations. Find the coefficient of determination and check whether r is significant using probable error.

Show the solution

r² = 0.6 × 0.6 = 0.36, so 36% of the variation in sales is explained by advertising spend.
Non-determination = 1 − 0.36 = 0.64, so 64% is unexplained.
1 − r² = 0.64 and √36 = 6.
PE = 0.6745 × 0.64 ÷ 6 = 0.43168 ÷ 6 = 0.07195, approximately 0.072.
6 × PE = 6 × 0.072 = 0.432 (approximately).
r = 0.6 is greater than 0.432, so r is significant.

Answer: r² = 0.36 (36% explained); PE ≈ 0.072; since r > 6 × PE, r is significant.

Exam tips

Questions are mostly direct: find PE, find r², or decide significance. Learn the formula exactly, especially (1 − r²) and √N.
Questions often give r² as a percentage. Convert to a decimal and take the square root to recover |r|.
Options often include a wrong answer from using 1 − r instead of 1 − r². Check you used r².
There is no negative marking, so always mark an answer. Eliminate options whose PE is clearly too large for the given N.

Practice questions from Correlation and Regression

Probable Error and Coefficient of Determination in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Probable Error and Coefficient of Determination: frequently asked questions

What is the formula for probable error of the correlation coefficient?

PE = 0.6745 × (1 − r²) ÷ √N, where r is the correlation coefficient and N is the number of pairs of observations. A smaller PE means r is more reliable.

What does the coefficient of determination mean?

It is r², the proportion of the variation in one variable that is explained by its linear relationship with the other. For example, r = 0.9 gives r² = 0.81, meaning 81% of the variation is explained.

How do I interpret r squared in correlation?

Multiply r² by 100 to get the percentage of variation explained. The rest, 1 − r², is explained by other factors. r² does not tell you whether the relationship is positive or negative.

When is r considered significant using probable error?

If |r| is more than 6 times the probable error, r is taken as significant. If it is less than 6 times PE, there is no real evidence of correlation. If r is less than PE itself, there is no evidence of correlation at all.