Suppose the true cause-and-effect relationship is given by: with parameters a, b, c, dependent variable y, independent variables x and z, and error termu. We wish to know the effect of x itself upon y. Two conditions must hold true for omitted-variable bias to exist in linear regression:
the omitted variable must be a determinant of the dependent variable ; and
the omitted variable must be correlated with an independent variable specified in the regression.
Suppose we omitz from the regression, and suppose the relation between x and z is given by with parameters d, f and error term e. Substituting the second equation into the first gives If a regression of y is conducted upon x only, this last equation is what is estimated, and the regression coefficient on x is actually an estimate of, giving not simply an estimate of the desired direct effect of x upon y, but rather of its sum with the indirect effect. Thus by omitting the variable z from the regression, we have estimated the total derivative of ywith respect tox rather than its partial derivative with respect to x. These differ if both c and f are non-zero. The direction and extent of the bias are both contained in cf, since the effect sought is b but the regression estimates b+cf. The extent of the bias is the absolute value of cf, and the direction of bias is upward if cf > 0, and it is downward otherwise.
Detailed analysis
As an example, consider a linear model of the form where
xi is a 1 × prow vector of values of p independent variables observed at time i or for the i th study participant;
β is a p × 1 column vector of unobservable parameters to be estimated;
zi is a scalar and is the value of another independent variable that is observed at time i or for the i th study participant;
δ is a scalar and is an unobservable parameter to be estimated;
ui is the unobservable error term occurring at time i or for the i th study participant; it is an unobserved realization of a random variable having expected value 0 ;
yi is the observation of the dependent variable at time i or for the i th study participant.
We collect the observations of all variables subscripted i = 1,..., n, and stack them one below another, to obtain the matrixX and the vectorsY, Z, and U: and If the independent variable z is omitted from the regression, then the estimated values of the response parameters of the other independent variables will be given by the usual least squares calculation, . Substituting for Y based on the assumed linear model, On taking expectations, the contribution of the final term is zero; this follows from the assumption that U is uncorrelated with the regressorsX. On simplifying the remaining terms: The second term after the equal sign is the omitted-variable bias in this case, which is non-zero if the omitted variable z is correlated with any of the included variables in the matrixX. Note that the bias is equal to the weighted portion of zi which is "explained" by xi.
The Gauss–Markov theorem states that regression models which fulfill the classical linear regression model assumptions provide the most efficient, linear and unbiased estimators. In ordinary least squares, the relevant assumption of the classical linear regression model is that the error term is uncorrelated with the regressors. The presence of omitted-variable bias violates this particular assumption. The violation causes the OLS estimator to be biased and inconsistent. The direction of the bias depends on the estimators as well as the covariance between the regressors and the omitted variables. A positive covariance of the omitted variable with both a regressor and the dependent variable will lead the OLS estimate of the included regressor's coefficient to be greater than the true value of that coefficient. This effect can be seen by taking the expectation of the parameter, as shown in the previous section.
