Pattern search (optimization)


Pattern search is a family of numerical optimization methods that does not require a gradient. As a result, it can be used on functions that are not continuous or differentiable. One such pattern search method is "convergence", which is based on the theory of positive bases. Optimization attempts to find the best match in a multidimensional analysis space of possibilities.

History

The name "pattern search" was coined by Hooke and Jeeves. An early and simple variant is attributed to Fermi and Metropolis when they worked at the Los Alamos National Laboratory. It is described by Davidon, as follows:

Convergence

Convergence is a pattern search method proposed by Yu, who proved that it converges using the theory of positive bases. Later, Torczon, Lagarias and co-authors used positive-basis techniques to prove the convergence of another pattern-search method on specific classes of functions. Outside of such classes, pattern search is a heuristic that can provide useful approximate solutions for some issues, but can fail on others. Outside of such classes, pattern search is not an iterative method that converges to a solution; indeed, pattern-search methods can converge to non-stationary points on some relatively tame problems.