## What conditions on a probability distribution defined by long-time averaging do I need to satisfy a central limit theorem?

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

For integer $n$, $1 \le n \le N$, consider the random variables

$X_n = \cos[t \omega_n]$

For any fixed $N$, we can take the mean

$Y_N = \frac{1}{N} \sum_{n=1}^N X_n$

and define a (cumulative) distribution by averaging over long times:

$P(Y_N \le y) = \lim_{T \to \infty} \frac{1}{2 T} \lambda(\{t \in [-T,T] \; \vert \; Y_N \le y \} )$,

where $\lambda$ is the Lebesgue measure. If the $\omega_n$ are linearly independent over the rationals for all $n$, then the random variables $X_n$ are independent and identically distributed over long times. Since they are i.i.d., we can apply the central limit theorem to show that $p(Y_N = y)$, the probability density of $Y_N$, approaches a normal distribution with zero mean and variance $\sigma_0^2/N$, where $\sigma_0^2=1/2$ is the variance of each $X_n$.

Now, suppose that the $\omega_n$ are not all linearly independent, but are incommensurate (i.e. pairwise linearly independent). This would mean that the $X_n$ are not independent. In particular, suppose we consider only $N=2^M$ for integer $M$ and take

$w_n^{(M)} = \sum_{k=0}^{M-1} (-1)^{r_k} h_k$

where

$r_k = (n/2^k) \mod 2$

are the digits of $n$ in base 2. Though the $h_k$ may be linearly independent over the rationals, there are $2^M$ frequencies $w_n^{(M)}$ for each fixed $M$. Since $k$ only ranges from $0$ to $M-1$, these frequencies are not linearly independent.

So for fixed $M$, not only are the $2^M$ random variables $X_n^{(M)} = \cos[t \omega_n^{(M)}]$ not independent, their joint probability density is actually only has support on a $M$-dimensional subspace. This is not a stationary sequence (I think). I'm not sure if it is ergodic.

Now, we could express the $Y$ as a mean of just $M$ independent random variables by combining the $X_n^{(M)}$. This would seem to guarantee a variance only as small as $\sigma_0^2/M$ rather than $\sigma_0^2/2^M$.

However, numerical work confirmed my intuition that the the $Y$ actually approaches a normal distribution with the small variance $\sigma_0^2/2^M$. This suggests there is a CLT I could apply to get this result analytically. But when I read the standard texts, I can't find much in the way of CLT's for non-stationary processes. I'm trying to read about ergodicity, but I can't even tell if this sequence fits the descriptions.

Is this sequence ergodic? Does it satisfy a CLT?

-

## 1 Answer

In your example, algebraic manipulation gives $$2^{-M} \sum_{n=0}^{2^M-1} X_n^{(M)} = \prod_{j=0}^{M-1} \cos t h_j.$$ If the $h_j$'s are linearly independent over $\Bbb Q$, then, as you point out, the random variables $\cos t h_j$ approach independence as $t$ is chosen over larger and larger intervals. Therefore, in the limit, $$Y = Z_1 ... Z_M, \qquad Z_i = \cos W_i, \qquad W_1,...,W_M {\rm\ \ i.i.d.\ uniform\ on\ } [0,2\pi).$$ By symmetry, ${\bf E}[Y]=0$, and since ${\bf E}[Z_i^2]=1/2$ for each $i$, we have ${\bf Var}[Y]=2^{-M}$. This may explain the result of your numerical experiments. However, $Y$ does not approach normal after rescaling: if $Y$ is rescaled to unit variance by setting $Y'_M:=2^{M/2} Y$, then $\log |Y'_M|=\log(\sqrt{2} |Z_1|)+...+\log(\sqrt{2} |Z_M|)$, so, since ${\bf E}[\log(\sqrt{2} |Z_i|)]<0$ and ${\bf Var}[\log(\sqrt{2} |Z_i|)]<\infty$, we can apply the CLT to $\log |Y'_M|$ to show that, as $M\to\infty$, $\log |Y'_M|$ will converge weakly to normal after rescaling, but $Y'_M$ converges weakly to 0.

-
Thanks very much for this compact and clear explanation. You are right about $Y$ not approaching a normal distribution. I discovered an an error in my code this morning, and the true distribution it approaches has much heavier tails than a Gaussian. Could you please elaborate on how you know that Var[$Y$] = $2^{-M}$? Is it correct to express Var[$Y$] in terms of the moments of the r.v. ln($Y$) (which is normally distributed), yielding a taylor series? Are there convergence issues I should worry about as a physicist? – Jess Riedel Sep 12 2011 at 17:19

1
To compute the variance of $Y$, observe that $Y^2$ is the product of $M$ independent random variables, each distributed as $\cos^2 W$, where $W$ is uniform. Since $\cos^2$ has average value $\frac{1}{2}$, and the r.v.s are independent, ${\bf E}[Y^2]$ is the product of $M$ copies of $\frac{1}{2}$, which is $2^{-M}$. – David Moews Sep 13 2011 at 1:07

1
Setting $Y_M:=Y$, there is no way to pick scaling constants $a_M$ such that $a_M Y_M$ converges to something nontrivial. $|Y_M|^{1/\sqrt{M}}$ will converge if rescaled appropriately. – David Moews Sep 13 2011 at 1:58
Ha, very embarrassing on my part, David. Good thing this is saved on the internet for perpetuity. Thanks so much for the help. – Jess Riedel Sep 14 2011 at 14:14
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 78, "mathjax_display_tex": 2, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9377729296684265, "perplexity_flag": "head"}