In numerical analysis, the speed at which a convergent sequence approaches its limit is represented by the order of convergence and the rate of convergence. A sequence that converges to is said to have order of convergence and rate of convergence if The value is also called the asymptotic error constant. Strictly speaking, the asymptotic behavior of a sequence does not, give conclusive information about any finite part of the sequence. In practice, however, the rate and order of convergence provide useful insights when using iterative methods for calculating approximations. If the order of convergence is higher, then typically fewer iterations are necessary to yield a useful approximation. This can make the difference between needing ten iterations or a million. Similar concepts are used for discretization methods. The solution of the discretized problem converges to the solution of the continuous problem as the grid size goes to zero, and the speed of convergence is one of the factors of the efficiency of the method. However, the terminology, in this case, is different from the terminology for iterative methods. Series acceleration is a collection of techniques for improving the rate of convergence of a series discretization. Such acceleration is commonly accomplished with sequence transformations.
Convergence speed for iterative methods
Q-convergence definitions
Suppose that the sequence converges to the number. The sequence is said to converge Q-linearly to if there exists a number such that The number is called the rate of convergence. The sequence is said to converge Q-superlinearly to if and it is said to converge Q-sublinearly to if If the sequence converges sublinearly and additionally then it is said that the sequence converges Q-logarithmically to. In order to further classify convergence, the order of convergence is defined as follows. The sequence is said to converge with order to for if for some positive constant . In particular, convergence with order
is called linear convergence,
is called quadratic convergence,
is called cubic convergence,
etc.
Some sources only define order of convergence for strictly greater than. In the definitions above, the "Q-" stands for "quotient" because the terms are defined using the quotient between two successive terms. Often, however, the "Q-" is dropped and a sequence is simply said to have linear convergence, quadratic convergence, etc.
Order estimation
A practical method to calculate the order of convergence for a sequence is to calculate the following sequence, which converges to
R-convergence definition
The drawback of the above definitions is that they do not catch some sequences which converge reasonably fast, but whose rate is variable, such as the sequence below. Therefore, the definition of rate of convergence is sometimes extended as follows. Under the new definition, the sequence converges with at least order if there exists a sequence such that and the sequence converges to zero with order according to the above "simple" definition. To distinguish it from that definition, this is sometimes called R-linear convergence, R-quadratic convergence, etc..
Examples
Consider the following sequences: The sequence converges linearly to 0 with rate 1/2. More generally, the sequence converges linearly with rate if. The sequence also converges linearly to 0 with rate 1/2 under the extended definition, but not under the simple definition. The sequence converges superlinearly. In fact, it is quadratically convergent. Finally, the sequence converges sublinearly and logarithmically.
Convergence speed for discretization methods
A similar situation exists for discretization methods. The important parameter here for the convergence speed is not the iteration number k, but the number of grid points and grid spacing. In this case, the number of grid points n in a discretization process is inversely proportional to the grid spacing. In this case, a sequence is said to converge to L with order p if there exists a constant C such that This is written as using big O notation. This is the relevant definition when discussing methods for numerical quadrature or the solution of ordinary differential equations. A practical method to calculate the rate of convergence for a discretization method is to implement the following formula: where and denote the errors w.r.t. the new and old step sizes and respectively.
Examples (continued)
The sequence with was introduced above. This sequence converges with order 1 according to the convention for discretization methods. The sequence with, which was also introduced above, converges with order p for every number p. It is said to converge exponentially using the convention for discretization methods. However, it only converges linearly using the convention for iterative methods. The order of convergence of a discretization method is related to its global truncation error.
Acceleration of convergence
Many methods exist to increase the rate of convergence of a given sequence, i.e. to transform a given sequence into one converging faster to the same limit. Such techniques are in general known as "series acceleration". The goal of the transformed sequence is to reduce the computational cost of the calculation. One example of series acceleration is Aitken's delta-squared process.
