# Why do we need Markov chain

**Next page:**Stationary initial distributions

**Upwards:**Ergodicity and Stationarity

**Previous page:**Estimation of the speed of convergence; Perron-Frobenius theorem & nbsp

**content**

Irreducibility and aperiodicity

**proof****Notice**-
- The property of accessibility is
- The quality of communicating is one
*Equivalence relation*because it applies- (a)
- (Reflexivity),
- (b)
- exactly when (Symmetry),
- (c)
- and imply that (Transitivity).

- This means in particular

**Examples**

In addition to the irreducibility, we need another property of the states, namely the so-called *Aperiodicity*in order to be able to characterize the ergodicity of Markov chains in a simple way.

**definition**

We now show that the periods and match if the states belong to the same equivalence class of communicating states. We use the notation if .

**Theorem 2.8**If the states communicate, then applies .

**proof**

**Corollary 2.5**The Markov chain be irreducible. Then all of the states of the same period.

To be able to show

- that the characterization of the ergodicity of Markov chains considered in Section 2.2.1 (cf.Theorem 2.4) is synonymous with their irreducibility and aperiodicity,
- we still need the following elementary proposition from
*Number theory*.

**Lemma 2.3**Be any, but fixed, natural number. Then there is a natural number , so that

**proof**

**Theorem 2.9**The transition matrix is quasi-positive if and only if is irreducible and aperiodic.

**proof**-
- We first assume that the transition matrix is irreducible and aperiodic.
- For each let's look at the crowd , whose greatest common divisor is due to the aperiodicity of equal is.
- From the inequalities in Corollary 2.2 it follows that and thus
(50)

- We show the crowd contains two consecutive numbers.
- Be now . Because of (50) then also holds and for any , in which
(51) - However, we show
- that there are then natural numbers there so the difference between and less than is.
- From (51) it follows that and thus for

- That is why the crowd contains two consecutive numbers.
- From (50) and from Lemma 2.3 it follows that for each a there so that
(52) - From this, from the irreducibility of and from the inequality (25) in Corollary 2.2, i.e., that it follows for every couple of states is a natural number there so that i.e., is quasi-positive.
- Conversely, the irreducibility and aperiodicity of quasi-positive transition matrices result directly from the definitions of these terms.

- We first assume that the transition matrix is irreducible and aperiodic.
**Notice**-
- A simple example of a Markov chain that
*Not*is irreducible,- can through the model of the
*Weather forecast*given, where and - If or , then the associated Markov chain is obviously not irreducible (and therefore also not ergodic because of Theorem 2.9).

- can through the model of the
- It is still possible that the linear system of equations
(53)

one (or infinitely many) probabilistic solutions owns. - We now give examples of Markov chains to the
*Not*are aperiodic.- Here are the random variables
*Not*by a stochastic recursion equation given the shape (14), so that the "increments" are independent and identically distributed random variables. - We only assume that the random variables are "conditionally independent" in the following sense.
- As shown in Section 2.1.3, however, one can always choose to Construct stochastically equivalent Markov chains, the increments of which are independent, cf. the construction principle considered in (17) - (19).

- Here are the random variables
- So be and arbitrary finite (or countably infinite) sets, be any function, and be or. Random variables,
- One can show (see exercise 4.2) that the sequence given recursively by (54) is a Markov chain whose transition matrix is given by if for each .

- A simple example of a Markov chain that
**example**- (
*Diffusion model*)

see P. Brémaud (1999)*Markov Chains*. Springer-Verlag, New York, p.76 **Notice**-
- The Ehrenfest diffusion model is a special case of the following class of Markov chains used in the literature
*Birth and death processes with two reflective barriers*to be named. - The state space becomes again here considered while the transition matrix is given by
(58)

in which , and for each . - The linear system of equations then has the form
- One can show (see exercise 4.3) that
- in which by the normalization condition is given, i.e. or.

- Because we assume that and for each holds true, birth and death processes are apparently irreducible with two reflective barriers.
- If in addition for a holds, then birth and death processes with two reflective barriers are also aperiodic (and thus ergodic because of Theorem 2.9), see exercise 4.3.

- The Ehrenfest diffusion model is a special case of the following class of Markov chains used in the literature

**Next page:**Stationary initial distributions

**Upwards:**Ergodicity and Stationarity

**Previous page:**Estimation of the speed of convergence; Perron-Frobenius theorem & nbsp

**content**Ursa Pantle 2003-09-29

- What is on the way for you
- What is the conn in a submarine
- Unmarried couples can have sex in hotels
- What is Hong Kong independence
- Are Asian cultures intrinsically racist
- Where are all the English Americans
- Can telomerase reverse aging
- How do I find valid voucher codes
- What is trigonometry
- How many pounds is a British stone
- Which will keep you from getting bored
- How do i speak english
- What are some unique vacation ideas
- Are Chinese more patriotic than Americans
- What do Singaporeans think of responsibility?
- Why is there no light in the room
- How does the financing work
- Record sessions with therapists
- Why is Louis Vuitton so expensive
- Why is soda so bad for you
- How can I cure cancer
- Which two languages are most related
- How do conservatives see liberals
- What is acute pancreatitis