## Motivation of Moment Generating Functions

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

What is the motivation of defining the mmoment generating function of a random variable $X$ as: $E[e^{tX}]$? I know that one can obtain the mean, second moment, etc.. after computing it. But what was the intuition in using $e^{x}$? Is it because its one-to-one and always increasing?

-

## 4 Answers

Take the definition of "generating function" for a sequence. Do it where the sequence is the sequence consisting of the moments of $X$. That's it.

-

### You can accept an answer to one of your own questions by clicking the check mark next to it. This awards 15 reputation points to the person who answered and 2 reputation points to you.

If X and Y are independent then $E[e^{t(X+Y)}] = E[e^{tX}] E[e^{tY}]$, so convolution corresponds to multiplication of the mgf's. Another reason: the moment generating function is actually a Fourier transform.

Now suppose $X_i$ are i.i.d. with zero mean, and define $Y_n = \sum_{i=1}^n X_i/\sqrt{n}$. Define $\phi(t) = E[e^{tX_1}]$. Then $E[e^{tY_n}] = \phi(t/\sqrt{n})^n$. Under reasonable assumptions, $\phi(t) = 1 + V[X_1]t^2/2 + O(t^3)$, and so $E[e^{tY_n}] = (1 + V[X_1]t^2/2n + O(t^3)/n^{1.5})^n \longrightarrow e^{V[X_1]t^2/2}$, and we get the central limit theorem (by continuity of the Fourier transform).

-

2
Actually, it's a Laplace transform. The difference is important: it's possible to find $X_1$ such that $\phi(t)$ diverges except for $t=0$. But the Fourier transform $E(e^{itX})$ is defined for every real-valued random variable and every real $t$. – Mark Meckes Aug 1 2010 at 3:30

As you suggest, the fact that $e^x$ is increasing is a useful property here. In particular, that lets you in some cases to apply Markov's/Chebychev's inequality to the random variable $e^{tX}$ in order to get exponentially decaying bounds on the tails of $X$; see e.g. Chernoff bounds. In principle any other positive increasing function could be used in the same way, but $e^x$ is a particularly useful choice because it is especially well suited to studying sums of independent random variables, as noted already in Yuval's answer.

-

I think this is an answer to "What is the use of moment-generating functions", and not to "Where does the definition of moment-generating functions come from?" – shreevatsa Aug 1 2010 at 11:56
Fair enough.  I'll try to turn this into an answer to the question that was actually asked. – Mark Meckes Aug 1 2010 at 13:45

The goal is to to put all the moments in one package. Since $$e^{tx} = \sum \frac{x^n}{n!} t^n$$ the coefficients of $t^n$ in $E(e^{tx})$ are (scaled) moments. In other contexts we can use $$(1-xt)^{-1} = \sum x^n t^n$$ in place of $e^{tx}$. This gives more or less what engineers call the "z-transform" and in combinatorics it is known as "ordinary generating function". Using the exponential has the happy advantage that convolution of random variables translates to product of moment generating functions.

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 23, "mathjax_display_tex": 2, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 20, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9343904852867126, "perplexity_flag": "head"}