Markov chains on a measurable state space


A Markov chain on a measurable state space is a discrete-time-homogenous Markov chain with a measurable space as state space.

History

The definition of Markov chains has evolved during the 20th century. In 1953 the term Markov chain was used for stochastic processes with discrete or continuous index set, living on a countable or finite state space, see Doob. or Chung. Since the late 20th century it became more popular to consider a Markov chain as a stochastic process with discrete index set, living on a measurable state space.

Definition

Denote with a measurable space and with a Markov kernel with source and target.
A stochastic process on is called a time homogeneous Markov chain with Markov kernel and start distribution if
is satisfied for any. One can construct for any Markov kernel and any probability measure an associated Markov chain.

Remark about Markov kernel integration

For any measure we denote for -integrable function the Lebesgue integral as. For the measure defined by we used the following notation:

Basic properties

Starting in a single point

If is a Dirac measure in, we denote for a Markov kernel with starting distribution the associated Markov chain as on and the expectation value
for a -integrable function. By definition, we have then
We have for any measurable function the following relation:

Family of Markov kernels

For a Markov kernel with starting distribution one can introduce a family of Markov kernels by
for and. For the associated Markov chain according to and one obtains

Stationary measure

A probability measure is called stationary measure of a Markov kernel if
holds for any. If on
denotes the Markov chain according to a Markov kernel with stationary measure, and the distribution of is, then all
have the same probability distribution, namely:
for any.

Reversibility

A Markov kernel is called reversible according to a probability measure if
holds for any.
Replacing shows that if is reversible according to, then must be a stationary measure of.