A simple problem in markov chains

Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

I'm trying to understand a 1954 paper of Kubo intitled "Note on the stochastic theory of resonance absorption". The specific problem can be stated mathematically as follows: let $X(t)$ be a random process taking $n$ positive real values ${\omega_1,...,\omega_n}$. Suppose that $X(t)$ is Markov and its probablility transitions $P_{ij}(t) := P(X(t)= \omega_j | X(0) = \omega_i)$ satisfy

$P'_{jk}(t)= -c_kP_{jk}(t)+\sum_m P_{jm}(t)c_m P_{mk}$,

where $p_{ii}=0$ and $\sum_j p_{ij} = 0$.

We want to find the expectation value of $M(t):=\exp(i\int_0^t X(t')dt')$ in terms of parameters $\omega_i$, $c_i$ and $p_{ij}$.

Kubo's strategy seems to condition on $X(0) = \omega_i \wedge X(t) = \omega_j$ so he can find a link to the $P_{ij}(t)$, but I don't understand this too much... specifically he introduce a funtion $Q_{ij}(t)$ which is the average of $M(t)$ on the condition that the process is in $\omega_i$ at time $t=0$ and is found in the state $\omega_k$ at time $t$. He concludes that

$Q'_{jk}(t) = (i\omega_k-c_k)Q_{jk} (t) + \sum_m Q _{jm}(t)c_m p_{mk}$

I can't figure how to get to this conclusion.

-

There should be an answer! The problem is so easily formulated: given the Markov process $X(t)$ taking $n$ positive real values, find the expectation value of $M(t):=exp(i\int_0^t X(t') dt')$. – The man in the box Nov 9 2010 at 9:06
Does the answer below correspond to what you were asking for? – Didier Piau Apr 11 2011 at 17:10

1 Answer

Indeed, these formulas are standard. Their derivation in a slighly more general setting than yours is as follows.

For every $t\ge0$, let $$M_t=\displaystyle\exp\left(\int_0^tv(X_s)\mathrm{d}s\right),$$ for a given function $v$ defined on the state space of the process $(X_t)$. (In your setting, every $X_t$ is real valued and $v(x)=\mathrm{i}x$ for every $x$ but these details are irrelevant.) For every states $x$ and $y$, let $$A_t(y)=[X_t=y],\qquad Q_t(x,y)=E(M_t1_{A_t(y)}\vert X_0=x).$$ Let $Q_t$ denote the associated square matrix (indexed by the state space, possibly infinite). For instance, $Q_0$ is the identity matrix. Note also that in the expression of $Q_t(x,y)$, $[X_0=x]$ appears as a conditioning while $A_t(y)=[X_t=y]$ is the event to which the expectation is restricted and that these are different operations hence your interpretation of Kubo's method should be rephrased.

The dynamics of $(Q_t)$ is driven by a linear differential equation $Q'_t=GQ_t$, where $G$ is a deformation of the infinitesimal generator of the process $(X_t)$. To identify $G$, one can compute $Q_{t+s}$ at the order $s$, for $s > 0$, when $s$ is small.

To do so, call $r(x,y)$ the transition rate of $(X_t)$ from $x$ to $y\ne x$, and $c(x)$ the sum over $y\ne x$ of $r(x,y)$. (In Kubo's setting as reproduced in your post, $c(x)$ is your $c_x$ and $r(x,y)$ is your $c_xp_{xy}$. By the way, the sum over $y\ne x$ of your $p_{xy}$ should be $1$ instead of $0$ and you should make up your mind between the notations $p_{xy}$ and $P_{xy}$.)

Then, conditioning on $[X_0=x]$, one can decompose the expectation which defines $Q_{t+s}(x,y)$ along the values of $X_s$. This decomposition goes as follows. For every $z\ne x$, $X_s=z$ with probability $r(x,z)s+o(s)$, and $X_s=x$ with probability $1-c(x)s+o(s)$. Furthermore, for every $z\ne x$, $M_{t+s}=(1+o(1))M_t$ on $[X_0=x,X_s=z]$. And on $[X_0=X_s=x]$, the probability of a double transition in the time interval $[0,s]$ is $o(s)$, hence $M_{t+s}=(1+v(x)s+o(s))M_{t+s}/M_s$ where $M_{t+s}/M_s$ is distributed like $M_t$ conditional on $[X_0=x]$.

All this leads to $$Q_{t+s}(x,y)=Q_t(x,y)(1+v(x)s)(1-c(x)s)+\sum_zQ_t(z,y)r(x,z)s+o(s).$$ When $s\to0$, one gets $$Q'_t(x,y)=(v(x)-c(x))Q_t(x,y)+\sum_zr(x,z)Q_t(z,y).$$ In other words, $G(x,x)=v(x)-c(x)$ for every $x$ and $G(x,y)=r(x,y)$ for every $y\ne x$. These are the equations in Kubo's paper.

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 86, "mathjax_display_tex": 4, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": false, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9448911547660828, "perplexity_flag": "head"}