## Markov chain convergence problem.

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

Consider a markov chain matrix P of size n x n (n states).

P is known to be:

1- there are at least two absorbent states. one of them is denoted by null. (thus, we have that P_null,null = 1)

2- For the set of states that are not absorbent (called set H) , we have that P_h,null > 0 for all h in H.

3- Not all states are recurrent.

4- Aperiodic (the return to some states can occur at irregular times).

It is true that limit when n goes to infinity of P^n converges? Is this result well known or is the proof simple?

Thanks.

-

## 2 Answers

Yes, uniqueness holds.

Condition 2 implies that every state $j$ is either absorbing $(j\not\in H)$ or transient $(j\in H)$. Define the absorption time to be $T=\inf (n\geq 0: X_n\not\in H)$. This $T$ is almost surely finite for any starting state $i$, that is, the chain is eventually absorbed.

If $j\in H$, then $p^n_{ij}=P_i(X_n=j)\leq P_i(T>n)\to 0=:Q_{ij}$ as $n\to\infty$.

If $j\not\in H$, then $p^n_{ij}=P_i(X_n=j)\uparrow P_i(X_T=j)=:Q_{ij}$ as $n\to\infty$.

-

### You can accept an answer to one of your own questions by clicking the check mark next to it. This awards 15 reputation points to the person who answered and 2 reputation points to you.

Yes. $P^n$ converges to a matrix $Q$ with

(i) $Q_{i,i}=1$ for each $i\not\in H$ ($i$ is absorbent), and

(ii) $\sum_{j\not\in H} Q_{i,j}=1$ for all $i\in H$.

To see (ii) we need that $\sum_{j\not\in H}P^n_{i,j}\rightarrow 1$ for all $i\in H$. For this note that $\sum_{j\not\in H}P^n_{i,j}$ is the probability of going from $i$ to an absorbent state in $n$ steps, and so if $P_{i,\text{null}}\ge\lambda>0$ for all $i\in H$ then for all $j\not\in H$, $$P^n_{i,j}\le (1-\lambda)^n\rightarrow 0.$$ To get a unique such $Q$ we need to show for each absorbent state (say, for null) $$\lim_n \ P^n_{i,\text{null}}\quad\text{exists}$$ for each $i\in H$. But $P^n_{i,\text{null}}\le P^{n+1}_{i,\text{null}}$ since once we get to null we stay there.

-

1
? Let's call another absorbent state foo. Can't we have in addition (1') $P_{i,\mathrm{foo}}>0$ for all $i \in H$ ?? – Gerald Edgar Sep 9 2010 at 18:15
@Gerald Edgar: You're right. I fixed my answer in response to your comment. – Bjørn Kjos-Hanssen Sep 9 2010 at 18:50
The proof of uniqueness is well done? Not clear to me. Thanks. – Gerardo Sep 9 2010 at 20:30
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 32, "mathjax_display_tex": 2, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9272111058235168, "perplexity_flag": "middle"}