Introduction To The Moment Generating Function
28 Dec 2017Prologue To The Moment Generating Function
What is a Moment?
The expected values $E\lbrack X\rbrack$, $E\lbrack X^2\rbrack$, $E\lbrack X^3\rbrack$,…, and $E\lbrack X^r\rbrack$ are called moments. As you might already have explored in the statistics related reference and have it that:
➀the mean, $\mu=E\lbrack X\rbrack$.
➁the variance, $\sigma^2=Var\lbrack X\rbrack=E\lbrack X^2\rbrack-E^2\lbrack X\rbrack$.They are called the functions of moments, sometimes are difficult to found. The moment generating function provides an add-in in finding the k-th ordinary moment, the mean, and even more.
What is an MGF?
The MGF(moment generating function) of a random variable X, say it discrete, is usually given by:
$M_X(t)=E\lbrack e^{t\cdot X}\rbrack$
$\;\;\;\;\;\;\;\;=E\lbrack 1+\frac{t\cdot X}{1!}+\frac{(t\cdot X)^2}{2!}+\cdots+\frac{(t\cdot X)^k}{k!}+\cdots\rbrack$
Where $E\lbrack e^{t\cdot X}\rbrack$ exists in $\lbrack -h, h\rbrack$ for some $h$.
$E\lbrack X\rbrack$, called the first ordinary moment of random variable $X$. More precisely, denoted by $\mu_1$.
$E\lbrack X^2\rbrack$, also called the second ordinary moment of random variable $X$, denoted by $\mu_2$.
$E\lbrack X^k\rbrack$, also called the k-th ordinary moment of random variable $X$, denoted by $\mu_k$.$e^{t\cdot X}=1$+$\frac{t\cdot X}{1!}$+$\frac{(t\cdot X)^2}{2!}$+$\cdots$+$\frac{(t\cdot X)^k}{k!}$+$\cdots$
By above Taylor series, the coefficient of $X^k$ is $\frac{t^k}{k!}$, hence, if $M_X(t)$ exists for $-h<t<h$, there must exist a mapping between the random variable $X$ and $e^{t\cdot X}$.If $p(X)$ and $p(Y)$ has $M_X(t)=M_Y(t)$, that is to say $E\lbrack e^{t\cdot X}\rbrack$=$\lbrack e^{t\cdot Y}\rbrack$,then, the distribution of random variable $X$ and $Y$ are the same.
The Deduction Of The $\mu_k$
There exists a lof many peoperties of the MGF, but the majority of this article focus on the way to find out the k-th ordinary moment. Next to deduce the discrete MGF to express the moment.
➀
$M_X(t)=E\lbrack e^{t\cdot X}\rbrack$
$\;\;\;\;\;\;\;\;=E\lbrack\sum_{k=0}^\infty\frac{(t\cdot X)^k}{k!}\rbrack$
$\;\;\;\;\;\;\;\;=\sum_{k=0}^\infty\frac{E\lbrack X^k\rbrack\cdot t^k}{k!}$
where $E\lbrack X^k\rbrack$=$\mu_k$=$\sum_{i=1}^{\infty}(x_i)^k\cdot p(x_i)$, and $X={x_1,x_2,x_3,\cdots}$,
$\;\;\;\;\;\;\;\;=\sum_{k=0}^\infty\frac{\mu_k\cdot t^k}{k!}$
$\;\;\;\;\;\;\;\;=\sum_{k=0}^\infty\frac{\lbrack\sum_{i=1}^{\infty}(x_i)^k\cdot p(x_i)\rbrack\cdot t^k}{k!}$
$\;\;\;\;\;\;\;\;=\sum_{i=1}^\infty\sum_{k=0}^{\infty}\frac{(x_i)^k\cdot t^k}{k!}\cdot p(x_i)$
$\;\;\;\;\;\;\;\;=\sum_{i=1}^\infty e^{t\cdot x_i}\cdot p(x_i)$➁
$E\lbrack e^{t\cdot X}\rbrack=\sum_{i=1}^\infty e^{t\cdot x_i}\cdot p(x_i)$
$\;\;\;\;\;\;\;\;=\sum_{i=1}^\infty(1$+$\frac{t\cdot x_i}{1!}$+$\frac{(t\cdot x_i)^2}{2!}$+$\cdots$+$\frac{(t\cdot x_i)^k}{k!}$+$\cdots)\cdot p(x_i)$
$\;\;\;\;\;\;\;\;=\sum_{i=1}^\infty 1\cdot p(x_i)$+$\sum_{i=1}^\infty\frac{t\cdot x_i}{1!}\cdot p(x_i)$+$\sum_{i=1}^\infty\frac{(t\cdot x_i)^2}{2!}\cdot p(x_i)$+$\cdots$+$\sum_{i=1}^\infty\frac{(t\cdot x_i)^k}{k!}\cdot p(x_i)$+$\cdots$
$\;\;\;\;\;\;\;\;=1$+$t\cdot\sum_{i=1}^\infty x_i\cdot p(x_i)$+$\frac{t^2}{2!}\cdot\sum_{i=1}^\infty (x_i)^2\cdot p(x_i)$+$\cdots$+$\frac{t^k}{k!}\cdot\sum_{i=1}^\infty (x_i)^k\cdot p(x_i)$+$\cdots$
$\;\;\;\;\;\;\;\;=1$+$t\cdot E\lbrack X\rbrack$+$\frac{t^2}{2!}\cdot E\lbrack X^2\rbrack$+$\cdots$+$\frac{t^k}{k!}\cdot E\lbrack X^k\rbrack$+$\cdots$
$\;\;\;\;\;\;\;\;=1$+$t\cdot \mu_1$+$\frac{t^2}{2!}\cdot \mu_2$+$\cdots$+$\frac{t^k}{k!}\cdot\mu_k$+$\cdots$Therefore, all order of moment could be contained within MGF, which is the lemma.
➂we have the k-th derivatives of $M_X(t)$ with respect to $t^k$:
$\frac{\operatorname d^k M_X(t)}{\operatorname dt^k}$=$\mu_k$+$t\cdot \mu_{k+1}$+$\frac{t^2}{2!}\cdot \mu_{k+2}$+$\cdots$, where $\frac{\operatorname d^k M_X(t)}{\operatorname dt^k}$=$\mu_k$, for $t=0$.
Therefore, we can say that for two random variables $X$, $Y$ having the same MGF, then, they have the same distributon.