In statistics, Cook's distance or Cook's D is a commonly used estimate of the influence of a data point when performing a least-squaresregression analysis. In a practical ordinary least squares analysis, Cook's distance can be used in several ways: to indicate influential data points that are particularly worth checking for validity; or to indicate regions of the design space where it would be good to be able to obtain more data points. It is named after the American statistician R. Dennis Cook, who introduced the concept in 1977.
Definition
Data points with large residuals and/or high leverage may distort the outcome and accuracy of a regression. Cook's distance measures the effect of deleting a given observation. Points with a large Cook's distance are considered to merit closer examination in the analysis. For the algebraic expression, first define where is the error term, is the coefficient matrix, is the number of covariates or predictors for each observation, and is the design matrix including a constant. The least squares estimator then is, and consequently the fitted values for the mean of are where is the projection matrix. The -th diagonal element of, given by, is known as the leverage of the -th observation. Similarly, the -th element of the residual vector is denoted by. Cook's distance of observation is defined as the sum of all the changes in the regression model when observation is removed from it where is the fitted response value obtained when excluding, and is the mean squared error of the regression model. Equivalently, it can be expressed using the leverage :
There are different opinions regarding what cut-off values to use for spotting highly influential points. Since Cook's distance is in the metric of an F distribution with and degrees of freedom, the median point can be used as a cut-off. Since this value is close to 1 for large, a simple operational guideline of has been suggested. Note that the Cook's distance measure does not always correctly identify influential observations.
Relationship to other influence measures (and interpretation)
can be expressed using the Leverage and the square of the internallyStudentized residual, as follows: The benefit in the last formulation is that it clearly shows the relationship between and to . If is large then it will increase. If is close to 0 than will be small, while if is close to 1 then will become very large. is related to DFFITS through the following relationship : can be interpreted as the distance one's estimates move within the confidence ellipsoid that represents a region of plausible values for the parameters. This is shown by an alternative but equivalent representation of Cook's distance in terms of changes to the estimates of the regression parameters between the cases, where the particular observation is either included or excluded from the regression analysis.
Software implementations
Many programs and statistics packages, such as R, Python, etc., include implementations of Cook's distance.
Language/Program
Function
Notes
R
cooks.distance
See
Extensions
High dimensional Influence Measure, is an alternative to Cook's distance for when . While the Cook’s distance quantifies the individual observation’s influence on the least squares regression coefficient estimate, the HIM measures the influence of an observation on the marginal correlations.
