How do you use residuals to test for model assumptions and specification errors?

1 What are residuals?

Residuals are the errors or deviations of the regression model from the actual data points. They can be calculated by subtracting the predicted value of the dependent variable from the observed value for each observation. Residuals can be positive or negative, depending on whether the model underestimates or overestimates the true value. Ideally, residuals should be small, random, and normally distributed around zero, indicating that the model is accurate and unbiased.

2 How to plot residuals?

One of the most common ways to visualize residuals is to plot them against the predicted values of the dependent variable or the independent variables. This can help to identify patterns, outliers, or trends in the residuals that may indicate problems with the model fit or the data. For example, a residual plot can show if the residuals are heteroscedastic, meaning that they have different variances across different levels of the predictors. This can violate the assumption of constant variance and affect the reliability of the regression coefficients and standard errors.

3 How to test for normality of residuals?

Another assumption of many regression methods is that the residuals are normally distributed. This can be checked by using a normal probability plot or a histogram of the residuals. A normal probability plot is a scatter plot of the ordered residuals against the expected values of a normal distribution. If the residuals are normal, they should follow a straight line. A histogram is a bar chart of the frequency of the residuals in different intervals. If the residuals are normal, they should have a bell-shaped curve.

4 How to test for autocorrelation of residuals?

Autocorrelation is the correlation of the residuals with themselves at different time lags. It can occur when the data are collected over time or have a seasonal or cyclical pattern. Autocorrelation can violate the assumption of independence of errors and affect the accuracy and efficiency of the regression estimates. One way to test for autocorrelation is to use a Durbin-Watson test, which measures the degree of autocorrelation between adjacent residuals. Another way is to use a correlogram, which plots the autocorrelation coefficients at different lags.

5 How to test for multicollinearity of predictors?

Multicollinearity is the correlation of the independent variables with each other. It can cause problems when the predictors are highly correlated, meaning that they share a lot of information and explain the same variation in the dependent variable. Multicollinearity can inflate the variance of the regression coefficients and make them unstable and unreliable. One way to test for multicollinearity is to use a variance inflation factor (VIF), which measures how much the variance of a coefficient is increased due to multicollinearity. Another way is to use a correlation matrix, which shows the pairwise correlations of the predictors.

6 How to improve the model fit and specification?

If the residuals show signs of violating the assumptions or having specification errors, such as omitted or irrelevant variables, nonlinear relationships, or interaction effects, there are some steps that can be taken to improve the model fit and specification. For instance, you can transform the dependent or independent variables to reduce skewness, heteroscedasticity, or nonlinearity. Additionally, you can add or remove variables to capture more or less variation in the dependent variable. Furthermore, creating dummy variables or interaction terms can account for categorical or moderating effects. Lastly, using a different regression method may be more suitable for the data and the research question.