Functional regression


Functional regression is a version of regression analysis when responses or covariates include functional data. Functional regression models can be classified into four types depending on whether the responses or covariates are functional or scalar: scalar responses with functional covariates, functional responses with scalar covariates, functional responses with functional covariates, and scalar or functional responses with functional and scalar covariates. In addition, functional regression models can be linear, partially linear, or nonlinear. In particular, functional polynomial models, functional single and multiple index models and functional additive models are three special cases of functional nonlinear models.

Functional linear models (FLMs)

Functional linear models are an extension of linear models. A linear model with scalar response and scalar covariates can be written as
where denotes the inner product in Euclidean space, and denote the regression coefficients, and is a random error with mean zero and finite variance. FLMs can be divided into two types based on the responses.

Functional linear models with scalar responses

Functional linear models with scalar responses can be obtained by replacing the scalar covariates and the coefficient vector in model by a centered functional covariate and a coefficient function with domain, respectively, and replacing the inner product in Euclidean space by that in Hilbert space Lp space|,
where here denotes the inner product in. One approach to estimating and is to expand the centered covariate and the coefficient function in the same functional basis, for example, B-spline basis or the eigenbasis used in the Karhunen-Loève expansion. Suppose is an orthonormal basis of. Expanding and in this basis,,, model becomes
For implementation, regularization is needed and can be done through truncation, penalization or penalization. In addition, a reproducing kernel Hilbert space approach can also be used to estimate and in model
Adding multiple functional and scalar covariates, model can be extended to
where are scalar covariates with, are regression coefficients for, respectively, is a centered functional covariate given by, is regression coefficient function for, and is the domain of and, for. However, due to the parametric component, the estimation methods for model cannot be used in this case and alternative estimation methods for model are available.

Functional linear models with functional responses

For a functional response with domain and a functional covariate with domain, two FLMs regressing on have been considered. One of these two models is of the form
where is still the centered functional covariate, and are coefficient functions, and is usually assumed to be a random process with mean zero and finite variance. In this case, at any given time, the value of, i.e.,, depends on the entire trajectory of. Model, for any given time, is an extension of multivariate linear regression with the inner product in Euclidean space replaced by that in. An estimating equation motivated by multivariate linear regression is
where, is defined as with for. Regularization is needed and can be done through truncation, penalization or penalization. Various estimation methods for model are available.

When and are concurrently observed, i.e.,, it is reasonable to consider a historical functional linear model, where the current value of only depends on the history of, i.e., for in model. A simpler version of the historical functional linear model is the functional concurrent model.

Adding multiple functional covariates, model can be extended to
where for, is a centered functional covariate with domain, and is the corresponding coefficient function with the same domain, respectively. In particular, taking as a constant function yields a special case of model
which is a FLM with functional responses and scalar covariates.

Functional concurrent models

Assuming that, another model, known as the functional concurrent model, sometimes also referred to as the varying-coefficient model, is of the form
where and are coefficient functions. Note that model assumes the value of at time, i.e.,, only depends on that of at the same time, i.e.,. Various estimation methods can be applied to model.

Adding multiple functional covariates, model can also be extended to
where are multiple functional covariates with domain and are the coefficient functions with the same domain.

Functional nonlinear models

Functional polynomial models

Functional polynomial models are an extension of the FLMs with scalar responses, analogous to extending linear regression to polynomial regression. For a scalar response and a functional covariate with domain, the simplest example of functional polynomial models is functional quadratic regression
where is the centered functional covariate, is a scalar coefficient, and are coefficient functions with domains and, respectively, and is a random error with mean zero and finite variance. By analogy to FLMs with scalar responses, estimation of functional polynomial models can be obtained through expanding both the centered covariate and the coefficient functions and in an orthonormal basis.

Functional single and multiple index models

A functional multiple index model is given by
Taking yields a functional single index model. However, for, this model is problematic due to curse of dimensionality. With and relatively small sample sizes, the estimator given by this model often has large variance. An alternative -component functional multiple index model can be expressed as
Estimation methods for functional single and multiple index models are available.

Functional additive models (FAMs)

Given an expansion of a functional covariate with domain in an orthonormal basis :, a functional linear model with scalar responses shown in model can be written as
One form of FAMs is obtained by replacing the linear function of, i.e.,, by a general smooth function,
where satisfies for. Another form of FAMs consists of a sequence of time-additive models:
where is a dense grid on with increasing size, and with a smooth function, for

Extensions

A direct extension of FLMs with scalar responses shown in model is to add a link function to create a generalized functional linear model by analogy to extending linear regression to generalized linear regression, of which the three components are:
  1. Linear predictor ;
  2. Variance function, where is the conditional mean;
  3. Link function connecting the conditional mean and the linear predictor through.