[Sponsors]
Home > Wiki > Introduction to turbulence/Statistical analysis/Estimation fro...

# Introduction to turbulence/Statistical analysis/Estimation from a finite number of realizations

### From CFD-Wiki

(Difference between revisions)
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|--------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Jola  (Talk | contribs)              |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | ()       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| (3 intermediate revisions not shown) |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 1:                              |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 1:  |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +        | {{Introduction to turbulence menu}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|                                      | == Estimators for averaged quantities ==                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |          | == Estimators for averaged quantities ==                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 67:                             |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 68: |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |          | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | Thus ''the variability of the stimator depends inversely on the number of independent realizations, <math>N</math>, and linearly on the relative fluctuation level of the random variable itself <math>\sigma_{x}/ X</math>''. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the <math>x</math> itself, the greater the fluctuation ( relative to the mean of <math>x</math>, <math>\left\langle x \right\rangle = X</math> ), then the more independent samples it will take to achieve a specified accuracy. | +        | Thus ''the variability of the estimator depends inversely on the number of independent realizations, <math>N</math>, and linearly on the relative fluctuation level of the random variable itself <math>\sigma_{x}/ X</math>''. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the <math>x</math> itself, the greater the fluctuation ( relative to the mean of <math>x</math>, <math>\left\langle x \right\rangle = X</math> ), then the more independent samples it will take to achieve a specified accuracy. |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      | '''Example:''' In a given ensemble the relative fluctuation level is 12% (i.e. <math>\sigma_{x}/ X = 0.12</math>). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |          | '''Example:''' In a given ensemble the relative fluctuation level is 12% (i.e. <math>\sigma_{x}/ X = 0.12</math>). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Line 78:                             |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 79: |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      | or <math>N \geq 144</math>.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |          | or <math>N \geq 144</math>.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| -                                    |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | {| class="toccolours" style="margin: 2em auto; clear: both; text-align:center;"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | |-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | | [[Statistical analysis in turbulence|Up to statistical analysis]] | [[Multivariate random variables|Back to multivariate random variables]] | [[Generalization to the estimator of any quantity|Forward to generalization to the estimator of any quantity]]                                                                                                                                                                                                                                                                                                                                                                                                                                                             |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | |}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                                      | {{Turbulence credit wkgeorge}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |          | {{Turbulence credit wkgeorge}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                                      |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |          |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -                                    | [[Category: Turbulence]]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | +        | {{Chapter navigation|Multivariate random variables|Generalization to the estimator of any quantity}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

## Latest revision as of 16:42, 31 August 2007

Nature of turbulence
Statistical analysis
Ensemble average  Probability  Multivariate random var ...  Estimation from a finite ...  Generalization to the esti ...
Reynolds averaged equation
Turbulence kinetic energy
Stationarity and homogeneity
Homogeneous turbulence
Wall bounded turbulent flows
Study questions ... template not finished yet!

## Estimators for averaged quantities

Since there can never an infinite number of realizations from which ensemble averages (and probability densities) can be computed, it is essential to ask: How many realizations are enough? The answer to this question must be sought by looking at the statistical properties of estimators based on a finite number of realization. There are two questions which must be answered. The first one is:

• Is the expected value (or mean value) of the estimator equal to the true ensemble mean? Or in other words, is yje estimator unbiased?

The second question is

• Does the difference between the and that of the true mean decrease as the number of realizations increases? Or in other words, does the estimator converge in a statistical sense (or converge in probability). Figure 2.9 illustrates the problems which can arise.

## Bias and convergence of estimators

A procedure for answering these questions will be illustrated by considerind a simple estimator for the mean, the arithmetic mean considered above, $X_{N}$. For $N$ independent realizations $x_{n}, n=1,2,...,N$ where $N$ is finite, $X_{N}$ is given by:

$X_{N}=\frac{1}{N}\sum^{N}_{n=1} x_{n}$

Figure 2.9 not uploaded yet

Now, as we observed in our simple coin-flipping experiment, since the $x_{n}$ are random, so must be the value of the estimator $X_{N}$. For the estimator to be unbiased, the mean value of $X_{N}$ must be true ensemble mean, $X$, i.e.

$\lim_{N\rightarrow\infty} X_{N} = X$

It is easy to see that since the operations of averaging adding commute,

$\begin{matrix} \left\langle X_{N} \right\rangle & = & \left\langle \frac{1}{N} \sum^{N}_{n=1} x_{n} \right\rangle \\ & = & \frac{1}{N} \sum^{N}_{n=1} \left\langle x_{n} \right\rangle \\ & = & \frac{1}{N} NX = X \\ \end{matrix}$

(Note that the expected value of each $x_{n}$ is just $X$ since the $x_{n}$ are assumed identically distributed). Thus $x_{N}$ is, in fact, an unbiased estimator for the mean.

The question of convergence of the estimator can be addressed by defining the square of variability of the estimator, say $\epsilon^{2}_{X_{N}}$, to be:

$\epsilon^{2}_{X_{N}}\equiv \frac{var \left\{ X_{N} \right\} }{X^{2}} = \frac{\left\langle \left( X_{N}- X \right)^{2} \right\rangle }{X^{2}}$

Now we want to examine what happens to $\epsilon_{X_{N}}$ as the number of realizations increases. For the estimator to converge it is clear that $\epsilon_{x}$ should decrease as the number of sample increases. Obviously, we need to examine the variance of $X_{N}$ first. It is given by:

$\begin{matrix} var \left\{ X_{N} \right\} & = & \left\langle X_{N} - X^{2} \right\rangle \\ & = & \left\langle \left[ \lim_{N\rightarrow\infty} \frac{1}{N} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle - X^{2}\\ \end{matrix}$

since $\left\langle X_{N} \right\rangle = X$ from the equation for $\langle X_{N} \rangle$ above. Using the fact that operations of averaging and summation commute, the squared summation can be expanded as follows:

$\begin{matrix} \left\langle \left[ \lim_{N\rightarrow\infty} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle & = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}} \sum^{N}_{n=1} \sum^{N}_{m=1} \left\langle \left( x_{n} - X \right) \left( x_{m} - X \right) \right\rangle \\ & = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}}\sum^{N}_{n=1}\left\langle \left( x_{n} - X \right)^{2} \right\rangle \\ & = & \frac{1}{N} var \left\{ x \right\} \\ \end{matrix}$

where the next to last step follows from the fact that the $x_{n}$ are assumed to be statistically independent samples (and hence uncorrelated), and the last step from the definition of the variance. It follows immediately by substitution into the equation for $\epsilon^{2}_{X_{N}}$ above that the square of the variability of the estimator, $X_{N}$, is given by:

$\begin{matrix} \epsilon^{2}_{X_{N}}& =& \frac{1}{N}\frac{var\left\{x\right\}}{X^{2}} \\ & = & \frac{1}{N} \left[ \frac{\sigma_{x}}{X} \right]^{2} \\ \end{matrix}$

Thus the variability of the estimator depends inversely on the number of independent realizations, $N$, and linearly on the relative fluctuation level of the random variable itself $\sigma_{x}/ X$. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the $x$ itself, the greater the fluctuation ( relative to the mean of $x$, $\left\langle x \right\rangle = X$ ), then the more independent samples it will take to achieve a specified accuracy.

Example: In a given ensemble the relative fluctuation level is 12% (i.e. $\sigma_{x}/ X = 0.12$). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?

AnswerUsing the equation for $\epsilon^{2}_{X_{N}}$ above, and taking $\epsilon_{X_{N}}=0.01$, it follows that:

$\left(0.01 \right)^{2} = \frac{1}{N}\left(0.12 \right)^{2}$

or $N \geq 144$.

## Credits

This text was based on "Lectures in Turbulence for the 21st Century" by Professor William K. George, Professor of Turbulence, Chalmers University of Technology, Gothenburg, Sweden.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 39, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8918430209159851, "perplexity_flag": "middle"}