Take, and . Then a Markov kernel is fully determined by the probability it assigns to a singleton set with for each : Now the random walk that goes to the right with probability and to the left with probability is defined by where is the Kronecker delta. The transition probabilities for the random walk are equivalent to the Markov kernel.
More generally take and both countable and. Again a Markov kernel is defined by the probability it assigns to singleton sets for each We define a Markov process by defining a transition probability where the numbers define a stochastic matrix i.e. We then define Again the transition probability, the stochastic matrix and the Markov kernel are equivalent reformulations.
Markov kernel defined by a kernel function and a measure
Let be a measure on, and a measurable function with respect to the product -algebra such that then i.e. the mapping defines a Markov kernel.. This example generalises the countable Markov process example where was the counting measure. Moreover it encompasses other important examples such as the convolution kernels, in particular the Markov kernels defined by the heat equation. The latter example includes the Gaussian kernel on with standard Lebesgue measure and
Measurable functions.
Take and arbitrary measurable spaces, and let be a measurable function. Now define i.e. Note that the indicator function is -measurable for all iff is measurable. This example allows us to think of a Markov kernel as a generalised function with a random rather than certain value.
As a less obvious example, take, and the real numbers with the standard sigma algebra of Borel sets. Then with i.i.d. random variables and where is the indicator function. For the simple case of coin flips this models the different levels of a Galton board.
Composition of Markov Kernels and the Markov Category
Given measurable spaces, and, and probability kernels and, we can define a composition by The composition is associative by Tonelli's theorem and the identity function considered as a Markov kernel is the unit for this composition. This composition defines the structure of a category on the measurable spaces with Markov kernels as morphisms first defined by Lawvere. The category has the empty set as initial object and the one point set as the terminal object.
Probability Space defined by Probability Distribution and a Markov Kernel
A probability measure on a measurable space is the same thing as a morphism in the Markov category also denoted by. By composition, a probability space and a probability kernel defines a probability space. It is concretely defined by
Properties
Semidirect product
Let be a probability space and a Markov kernel from to some. Then there exists a unique measure on , such that:
Let be a Borel space, a -valued random variable on the measure space and a sub--algebra. Then there exists a Markov kernel from to, such that is a version of the conditional expectation for every, i.e. It is called regular conditional distribution of given and is not uniquely defined.
Generalizations
s generalize Markov kernels in the sense that for all, the map can be any type of measure, not necessarily a probability measure.
