On continuous Markov chains: statistics of recurrent states

Given a continuous Markov chains (and given the transition rates between the states) I would like to know the following:

1. mean time of permanence for all states.
2. higher order moments (i.e., variance and, possibly, CDF?)

In particular, I am also interested about computing the above quantities in the case when the system has to stay at least for a fixed amount of time in every state. For example, suppose there are for states {A, B, C, D}. When the system is in state A, it has to stay there for at least $T_A$ seconds, and the same for state B ($T_B$ seconds) and so on for all remaining states. In this case, is it still possible to determine the quantities in the list above, namely, mean permanence time for all states, variance and CDF?

-

1 Answer

If it has to stay at least a fixed amount of time in a state, it's no longer a continuous Markov chain: the future depends not only on the present but also on the past (namely how long you've been in the current state). However, if by "permanence time" you mean the amount of time it is in the state $A$ from when it enters until it first leaves that state, that will simply be $T_A + T$ where $T$ has an exponential distibution with rate $v_A$ (the rate at which the system makes transitions out of state $A$ after the initial period is up).

-

do you know how to compute the time it takes on average to come back to a certain state when the system leaves it? – Bob Dec 27 '11 at 16:18
Let $u_i$ be the expected time to reach state $A$ after , leaving state $i$.  By a "first-step analysis", you get a system of linear equations for the $u_i$, which you can solve. – Robert Israel Dec 27 '11 at 17:59
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 11, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": false, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9533785581588745, "perplexity_flag": "head"}