What is the ergodic Markov chain


Next page:Estimation of the speed of convergence; Perron-Frobenius theorem Upwards:Ergodicity and Stationarity Previous page:Ergodicity and Stationarity & nbsp content


Basic definitions and quasi-positive transition matrices


This leads to the following concept of Ergodicity of Markov chains.

definition
The Markov chain with the transition matrix or the associated -stepped transition matrices ) called ergodicif the limits
(33)

  1. for each exist,
  2. are positive and not of depend,
  3. a probability function form, i.e. .
example
(Weather forecast)


The ergodicity of Markov chains with general (finite) state space can be characterized with the help of the following term from the theory of positive matrices.

definition
 
Notice
If is a stochastic matrix for which it is a natural number there, so that all entries of are positive, then all entries are also from For each natural number positive.
proof
 
  • We first show that the condition
    (35)

    for a sufficient for the ergodicity of is.
    • Be and . The Chapman-Kolmogorov equation (23) then shows that
      and thus
      i.e., for each , where we put. In the same way it turns out that for each applies.
    • To the existence of the limit values To prove in (33), it suffices to show that for each
      (36)

    • For this we consider for arbitrary, but firmly given states the amounts and .
    • Be . Then applies
      and
    • By applying the Chapman-Kolmogorov equation (23) again, it follows that for any and

    • It follows that and (by induction) that for each
      (37)

    • So there is an (infinitely growing) sequence of natural numbers such that for each
      (38)

    • Because the differences monotonous in (38) holds for each (infinitely growing) sequence of natural numbers.
    • This proves the validity of (36).
  • The limit values are positive because
  • Also applies , because the interchange of limit and finite Summation is allowed.
  • The necessity of condition (35) follows directly from this and (33), taking into account that the state space is finite.

Notice
 
  • Because the limits Ergodic Markov chains not from depend, results from this and from the finiteness of the state space that
  • In the proof of Theorem 2.4, not only was the existence of the limit values shown, but the following Estimation of the speed of convergence derived: From (37) it follows that
    (39)

    or.
    (40)

    in which the integer part of designated.
  • In connection with (39) and (40) one speaks in the literature of geometric bounds the speed of convergence.


We now show that the limit values can also be understood as the solution of a linear system of equations.

proof
 
Notice
 


Next page:Estimation of the speed of convergence; Perron-Frobenius theorem Upwards:Ergodicity and Stationarity Previous page:Ergodicity and Stationarity & nbsp content Ursa Pantle 2003-09-29