[Sponsors]
Home > Wiki > Introduction to turbulence/Statistical analysis/Estimation fro...

# Introduction to turbulence/Statistical analysis/Estimation from a finite number of realizations

### From CFD-Wiki

(Difference between revisions)
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|-------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Jola  (Talk | contribs) |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Jola  (Talk | contribs) |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Line 12:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 12:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | A procedure for answering these questions will be illustrated by considerind a simple '''estimator''' for the mean, the arithmetic mean considered above, <math>X_{N}</math>. For <math>N</math> independent realizations <math>x_{n}, n=1,2,...,N</math> where <math>N</math> is finite, <math>X_{N}</math> is given by:                                                                                                                                                                                                                                                                                                                                                                                                  |                         | A procedure for answering these questions will be illustrated by considerind a simple '''estimator''' for the mean, the arithmetic mean considered above, <math>X_{N}</math>. For <math>N</math> independent realizations <math>x_{n}, n=1,2,...,N</math> where <math>N</math> is finite, <math>X_{N}</math> is given by:                                                                                                                                                                                                                                                                                                                                                                                                  |
| -                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | +                       | :<math>X_{N}=\frac{1}{N}\sum^{N}_{n=1} x_{n}</math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| -                       | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | X_{N}=\frac{1}{N}\sum^{N}_{n=1} x_{n}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | <font color="orange" size="3">Figure 2.9 not uploaded yet</font>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |                         | <font color="orange" size="3">Figure 2.9 not uploaded yet</font>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| Line 23:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 19:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | Now, as we observed in our simple coin-flipping experiment, since the <math>x_{n}</math> are random, so must be the value of the estimator <math>X_{N}</math>. For the estimator to be ''unbiased'', the mean value of <math>X_{N}</math> must be true ensemble mean, <math>X</math>, i.e.                                                                                                                                                                                                                                                                                                                                                                                                                                 |                         | Now, as we observed in our simple coin-flipping experiment, since the <math>x_{n}</math> are random, so must be the value of the estimator <math>X_{N}</math>. For the estimator to be ''unbiased'', the mean value of <math>X_{N}</math> must be true ensemble mean, <math>X</math>, i.e.                                                                                                                                                                                                                                                                                                                                                                                                                                 |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | +                       | :<math>\lim_{N\rightarrow\infty} X_{N} = X</math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| -                       | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | \lim_{N\rightarrow\infty} X_{N} = X                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | It is easy to see that since the operations of averaging adding commute,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |                         | It is easy to see that since the operations of averaging adding commute,                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 39:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 30:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | (Note that the expected value of each <math>x_{n}</math> is just <math>X</math> since the <math>x_{n}</math> are assumed identically distributed). Thus <math>x_{N}</math> is, in fact, an ''unbiased estimator for the mean''.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         | (Note that the expected value of each <math>x_{n}</math> is just <math>X</math> since the <math>x_{n}</math> are assumed identically distributed). Thus <math>x_{N}</math> is, in fact, an ''unbiased estimator for the mean''.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| Line 45:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 35:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | The question of ''convergence'' of the estimator can be addressed by defining the square of '''variability of the estimator''', say <math>\epsilon^{2}_{X_{N}}</math>, to be:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                         | The question of ''convergence'' of the estimator can be addressed by defining the square of '''variability of the estimator''', say <math>\epsilon^{2}_{X_{N}}</math>, to be:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \epsilon^{2}_{X_{N}}\equiv \frac{var \left\{ X_{N} \right\} }{X^{2}} = \frac{\left\langle  \left( X_{N}- X \right)^{2} \right\rangle }{X^{2}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                         | \epsilon^{2}_{X_{N}}\equiv \frac{var \left\{ X_{N} \right\} }{X^{2}} = \frac{\left\langle  \left( X_{N}- X \right)^{2} \right\rangle }{X^{2}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | Now we want to examine what happens to <math>\epsilon_{X_{N}}</math> as the number of realizations increases. For the estimator to converge it is clear that <math>\epsilon_{x}</math> should decrease as the number of sample increases. Obviously, we need to examine the variance of <math>X_{N}</math> first. It is given by:                                                                                                                                                                                                                                                                                                                                                                                          |                         | Now we want to examine what happens to <math>\epsilon_{X_{N}}</math> as the number of realizations increases. For the estimator to converge it is clear that <math>\epsilon_{x}</math> should decrease as the number of sample increases. Obviously, we need to examine the variance of <math>X_{N}</math> first. It is given by:                                                                                                                                                                                                                                                                                                                                                                                          |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 60:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 47:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | since <math>\left\langle X_{N} \right\rangle = X</math> from equation 2.46. Using the fact that operations of averaging and summation commute, the squared summation can be expanded as follows:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | +                       | since <math>\left\langle X_{N} \right\rangle = X</math> from the equation for <math>\langle X_{N} \rangle</math> above. Using the fact that operations of averaging and summation commute, the squared summation can be expanded as follows:                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 72:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 57:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | where the next to last step follows from the fact that the <math>x_{n}</math> are assumed to be statistically independent samples (and hence uncorrelated), and the last step from the definition of the variance. It follows immediately by substitution into equation 2.49 that the square of the variability of the estimator, <math>X_{N}</math>, is given by:                                                                                                                                                                                                                                                                                                                                                         | +                       | where the next to last step follows from the fact that the <math>x_{n}</math> are assumed to be statistically independent samples (and hence uncorrelated), and the last step from the definition of the variance. It follows immediately by substitution into the equation for <math>\epsilon^{2}_{X_{N}}</math> above that the square of the variability of the estimator, <math>X_{N}</math>, is given by:                                                                                                                                                                                                                                                                                                              |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |                         | \begin{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| Line 83:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 66:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         | \end{matrix}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | Thus ''the variability of the stimator depends inversely on the number of independent realizations, <math>N</math>, and linearly on the relative fluctuation level of the random variable itself <math>\sigma_{x}/ X</math>''. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the <math>x</math> itself, the greater the fluctuation ( relative to the mean of <math>x</math>, <math>\left\langle x \right\rangle = X</math> ), then the more independent samples it will take to achieve a specified accuracy. |                         | Thus ''the variability of the stimator depends inversely on the number of independent realizations, <math>N</math>, and linearly on the relative fluctuation level of the random variable itself <math>\sigma_{x}/ X</math>''. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the <math>x</math> itself, the greater the fluctuation ( relative to the mean of <math>x</math>, <math>\left\langle x \right\rangle = X</math> ), then the more independent samples it will take to achieve a specified accuracy. |
| Line 89:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | Line 71:                |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | '''Example:''' In a given ensemble the relative fluctuation level is 12% (i.e. <math>\sigma_{x}/ X = 0.12</math>). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |                         | '''Example:''' In a given ensemble the relative fluctuation level is 12% (i.e. <math>\sigma_{x}/ X = 0.12</math>). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | '''Answer'''Using equation 2.52, and taking <math>\epsilon_{X_{N}}=0.01</math>, it follows that:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | +                       | '''Answer'''Using the equation for <math>\epsilon^{2}_{X_{N}}</math> above, and taking <math>\epsilon_{X_{N}}=0.01</math>, it follows that:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| -                       | <table width="100%"><tr><td>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | :<math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
|                         | \left(0.01 \right)^{2} = \frac{1}{N}\left(0.12 \right)^{2}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |                         | \left(0.01 \right)^{2} = \frac{1}{N}\left(0.12 \right)^{2}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
|                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |                         | </math>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -                       | </td><td width="5%">(2)</td></tr></table>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         | or <math>N \geq 144</math>.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |                         | or <math>N \geq 144</math>.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | {| class="toccolours" style="margin: 2em auto; clear: both; text-align:center;"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | |-                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | | [[Statistical analysis in turbulence|Up to statistical analysis]] | [[Multivariate random variables|Back to multivariate random variables]] | [[Generalization to the estimator of any quantity|Forward to generalization to the estimator of any quantity]]                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | |}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | {{Turbulence credit wkgeorge}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
|                         |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | +                       | [[Category: Turbulence]]                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

## Estimators for averaged quantities

Since there can never an infinite number of realizations from which ensemble averages (and probability densities) can be computed, it is essential to ask: How many realizations are enough? The answer to this question must be sought by looking at the statistical properties of estimators based on a finite number of realization. There are two questions which must be answered. The first one is:

• Is the expected value (or mean value) of the estimator equal to the true ensemble mean? Or in other words, is yje estimator unbiased?

The second question is

• Does the difference between the and that of the true mean decrease as the number of realizations increases? Or in other words, does the estimator converge in a statistical sense (or converge in probability). Figure 2.9 illustrates the problems which can arise.

## Bias and convergence of estimators

A procedure for answering these questions will be illustrated by considerind a simple estimator for the mean, the arithmetic mean considered above, $X_{N}$. For $N$ independent realizations $x_{n}, n=1,2,...,N$ where $N$ is finite, $X_{N}$ is given by:

$X_{N}=\frac{1}{N}\sum^{N}_{n=1} x_{n}$

Figure 2.9 not uploaded yet

Now, as we observed in our simple coin-flipping experiment, since the $x_{n}$ are random, so must be the value of the estimator $X_{N}$. For the estimator to be unbiased, the mean value of $X_{N}$ must be true ensemble mean, $X$, i.e.

$\lim_{N\rightarrow\infty} X_{N} = X$

It is easy to see that since the operations of averaging adding commute,

$\begin{matrix} \left\langle X_{N} \right\rangle & = & \left\langle \frac{1}{N} \sum^{N}_{n=1} x_{n} \right\rangle \\ & = & \frac{1}{N} \sum^{N}_{n=1} \left\langle x_{n} \right\rangle \\ & = & \frac{1}{N} NX = X \\ \end{matrix}$

(Note that the expected value of each $x_{n}$ is just $X$ since the $x_{n}$ are assumed identically distributed). Thus $x_{N}$ is, in fact, an unbiased estimator for the mean.

The question of convergence of the estimator can be addressed by defining the square of variability of the estimator, say $\epsilon^{2}_{X_{N}}$, to be:

$\epsilon^{2}_{X_{N}}\equiv \frac{var \left\{ X_{N} \right\} }{X^{2}} = \frac{\left\langle \left( X_{N}- X \right)^{2} \right\rangle }{X^{2}}$

Now we want to examine what happens to $\epsilon_{X_{N}}$ as the number of realizations increases. For the estimator to converge it is clear that $\epsilon_{x}$ should decrease as the number of sample increases. Obviously, we need to examine the variance of $X_{N}$ first. It is given by:

$\begin{matrix} var \left\{ X_{N} \right\} & = & \left\langle X_{N} - X^{2} \right\rangle \\ & = & \left\langle \left[ \lim_{N\rightarrow\infty} \frac{1}{N} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle - X^{2}\\ \end{matrix}$

since $\left\langle X_{N} \right\rangle = X$ from the equation for $\langle X_{N} \rangle$ above. Using the fact that operations of averaging and summation commute, the squared summation can be expanded as follows:

$\begin{matrix} \left\langle \left[ \lim_{N\rightarrow\infty} \sum^{N}_{n=1} \left( x_{n} - X \right) \right]^{2} \right\rangle & = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}} \sum^{N}_{n=1} \sum^{N}_{m=1} \left\langle \left( x_{n} - X \right) \left( x_{m} - X \right) \right\rangle \\ & = & \lim_{N\rightarrow\infty}\frac{1}{N^{2}}\sum^{N}_{n=1}\left\langle \left( x_{n} - X \right)^{2} \right\rangle \\ & = & \frac{1}{N} var \left\{ x \right\} \\ \end{matrix}$

where the next to last step follows from the fact that the $x_{n}$ are assumed to be statistically independent samples (and hence uncorrelated), and the last step from the definition of the variance. It follows immediately by substitution into the equation for $\epsilon^{2}_{X_{N}}$ above that the square of the variability of the estimator, $X_{N}$, is given by:

$\begin{matrix} \epsilon^{2}_{X_{N}}& =& \frac{1}{N}\frac{var\left\{x\right\}}{X^{2}} \\ & = & \frac{1}{N} \left[ \frac{\sigma_{x}}{X} \right]^{2} \\ \end{matrix}$

Thus the variability of the stimator depends inversely on the number of independent realizations, $N$, and linearly on the relative fluctuation level of the random variable itself $\sigma_{x}/ X$. Obviously if the relative fluctuation level is zero (either because there the quantity being measured is constant and there are no measurement errors), then a single measurement will suffice. On the other hand, as soon as there is any fluctuation in the $x$ itself, the greater the fluctuation ( relative to the mean of $x$, $\left\langle x \right\rangle = X$ ), then the more independent samples it will take to achieve a specified accuracy.

Example: In a given ensemble the relative fluctuation level is 12% (i.e. $\sigma_{x}/ X = 0.12$). What is the fewest number of independent samples that must be acquired to measure the mean value to within 1%?

AnswerUsing the equation for $\epsilon^{2}_{X_{N}}$ above, and taking $\epsilon_{X_{N}}=0.01$, it follows that:

$\left(0.01 \right)^{2} = \frac{1}{N}\left(0.12 \right)^{2}$

or $N \geq 144$.

## Credits

This text was based on "Lectures in Turbulence for the 21st Century" by Professor William K. George, Professor of Turbulence, Chalmers University of Technology, Gothenburg, Sweden.
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 39, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.8258053064346313, "perplexity_flag": "middle"}