Extensions of Multiple Regression for CFA Level II

What this chapter covers

Extensions of Multiple Regression: topics in the order to study them

1. 1Influence Analysis: Outliers and LeverageIt is about the data going into the model, so it comes first. It also introduces residual-based diagnostics you will reuse.
2. 2Dummy Variables in Multiple RegressionThis is a simple tool for building models, and interpreting intercept and slope shifts is needed before the problems that follow.
3. 3HeteroskedasticityThis is the first assumption violation. It introduces the pattern of detect, consequence, then correct that every later violation follows.
4. 4Serial CorrelationIt follows the same pattern as heteroskedasticity, so you can compare the two, including the tests and the robust standard error fix.
5. 5MulticollinearityIt differs from the previous two because it affects the standard errors of correlated variables rather than the residual pattern. Learn it after you have the other two clear.
6. 6Model MisspecificationIt pulls the earlier problems together and asks what went wrong in model design, so it is best studied once you know each individual violation.
7. 7Qualitative Dependent Variables: Probit, Logit, DiscriminantIt changes the type of dependent variable, so it comes last as a separate extension of the framework.

How to prepare Extensions of Multiple Regression

1. Refresh the basic regression output: coefficient, standard error, t-statistic, p-value, F-test and adjusted R². Everything later depends on reading these quickly.
2. For each violation, write a three-line card: how to detect it, what it does to coefficients and standard errors, and how to correct it. Keep the cards short enough to read on your phone.
3. Build a comparison table in your notes for heteroskedasticity, serial correlation and multicollinearity. Include the effect on the t-statistics and whether the coefficient estimates stay unbiased.
4. Practise dummy variables by writing the fitted equation for each category. Check how many dummies you need for n categories, and that the interpretation is relative to the omitted category.
5. Work through vignette-style questions. Underline the test statistic, the critical value or p-value, and the decision in the exhibit before you read the answer choices.
6. Finish with logit, probit and discriminant analysis. Focus on what the dependent variable is, what the output means, and how they differ from ordinary regression.
7. In the last week, redo your missed questions and recite your comparison table from memory.

Common mistakes in Extensions of Multiple Regression

• Saying heteroskedasticity or serial correlation biases the coefficient estimates.

  Fix: Remember that these two leave the coefficients unbiased in the usual setup but make the standard errors, and hence the t-tests, unreliable.

• Using n dummy variables for n categories.

  Fix: Use n − 1 dummies and read each coefficient relative to the omitted base category.

• Diagnosing multicollinearity from a single insignificant t-statistic.

  Fix: Look for a significant F-test and high R² alongside insignificant individual coefficients, or a high variance inflation factor.

• Confusing outliers with high-leverage observations.

  Fix: Tie outliers to the dependent variable and leverage to the independent variables. Then ask whether the point actually changes the fitted model.

• Matching the wrong remedy to the problem, such as robust errors for multicollinearity.

  Fix: Keep the three-line cards and the comparison table, and always read the fix next to its problem.

• Treating logit or probit output as a linear prediction of the outcome.

  Fix: Remember that these models estimate the probability of an event, and the coefficients are not read as simple changes in the outcome.

Last-day revision: Extensions of Multiple Regression

• ✓Outliers are extreme values of the dependent variable. High-leverage points are extreme values of the independent variables. Either can be influential.
• ✓With n categories, use n − 1 dummy variables. The omitted category is the base, and each dummy coefficient is a difference from it.
• ✓Heteroskedasticity means the error variance is not constant. Conditional heteroskedasticity is the problematic type, since it relates to the independent variables.
• ✓Heteroskedasticity leaves coefficient estimates unbiased but makes standard errors unreliable, so t-tests mislead.
• ✓The Breusch–Pagan test detects heteroskedasticity. A fix is to use robust (White-corrected) standard errors.
• ✓Serial correlation means errors are correlated across observations. Positive serial correlation typically understates standard errors and overstates t-statistics.
• ✓The Durbin–Watson test and the Breusch–Godfrey test detect serial correlation. Use Newey-West-type adjusted standard errors to correct it.
• ✓Multicollinearity means independent variables are highly correlated. The classic sign is a high R² and a significant F-test with insignificant individual t-statistics.
• ✓A variance inflation factor above a commonly used threshold (such as 5 or 10) signals concern about multicollinearity. Fixes include dropping or combining variables.
• ✓Misspecification includes omitted variables, wrong functional form, inappropriate scaling, and pooling data that should not be pooled. It can bias coefficients.
• ✓Logit and probit are used when the dependent variable is binary. Logit uses the logistic distribution and probit uses the normal distribution.
• ✓Discriminant analysis produces a score that classifies observations into categories.

Extensions of Multiple Regression in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

• FRM Part ILinear Regression
• Actuarial Core Principles (IAI)Graduation and graduation tests