Test Questions on Cook's Distance and Regression Influence Diagnostics

📚 Quick Study Guide

• 📈 Cook's Distance: A measure of how much the regression coefficients change when the $i$-th observation is removed. A high Cook's distance indicates a data point is influential.
• 📏 Formula for Cook's Distance: $D_i = \frac{\sum_{j=1}^{n} (\hat{Y_j} - \hat{Y_{j(i)}})^2}{p \cdot MSE}$, where $\hat{Y_j}$ is the predicted value for observation $j$, $\hat{Y_{j(i)}}$ is the predicted value for observation $j$ when observation $i$ is removed, $p$ is the number of predictors, and $MSE$ is the mean squared error.
• ⚠️ Cutoff for Cook's Distance: A common rule of thumb is that an observation with Cook's distance greater than 1 is considered influential. Another cutoff is $4/n$, where n is the number of observations.
• 🧪 Influence: An influential point is one that, if removed, would substantially change the regression results. Influential points may be outliers, but not all outliers are influential.
• 📊 Leverage: Leverage measures how far an observation's values on the independent variables are from the average of the independent variables. High leverage points have the potential to be influential.
• 🧩 Outliers: Outliers are data points that have large residual values (i.e., the observed value is very different from the predicted value).
• 📝 DFFITS: Measures the difference in the predicted value for each observation when that observation is excluded from the model.

Practice Quiz

1. Which of the following best describes Cook's Distance?
   1. A measure of multicollinearity.
   2. A measure of heteroscedasticity.
   3. A measure of how much the regression coefficients change when an observation is removed.
   4. A measure of the overall fit of the regression model.
2. What is a common cutoff value for Cook's Distance to indicate an influential point?
   1. 0.5
   2. 1
   3. 0.05
   4. 0.25
3. What does a high leverage value indicate?
   1. The observation has a small residual.
   2. The observation is close to the average of the independent variables.
   3. The observation's values on the independent variables are far from the average.
   4. The observation is an outlier in the dependent variable.
4. What is the primary goal of regression influence diagnostics?
   1. To improve the $R^2$ value.
   2. To identify data points that disproportionately affect the regression results.
   3. To correct for multicollinearity.
   4. To transform the data to meet regression assumptions.
5. Which of the following is NOT a common tool for regression influence diagnostics?
   1. Cook's Distance
   2. DFFITS
   3. Variance Inflation Factor (VIF)
   4. Leverage Values
6. How does the removal of an influential point typically affect the regression model?
   1. It always decreases the $R^2$ value.
   2. It always increases the $R^2$ value.
   3. It can significantly change the regression coefficients.
   4. It has no effect on the regression model.
7. What is the role of outliers in regression influence diagnostics?
   1. Outliers are always influential points.
   2. Outliers are never influential points.
   3. Outliers may be influential points and should be investigated.
   4. Outliers are only a concern in non-linear regression.