We are going to look at finding **probability generating functions**, **PGF**s, of discrete probability distributions, which can give us an efficient way to find expected values and variance, as well as allowing for greater analysis.

Let X represent a discrete random variable, which can take values x_{i} for i from 1 to n, with probabilities as per the following table:

x | x_{1} | x_{2} | x_{3} | … | x_{n} |

P(X=x) | P(X=x_{1}) | P(X=x_{2}) | P(X=x_{3}) | … | P(X=x_{n}) |

Using the table we can form the following function:

G_{X}(t) = P(X=x_{1})t^{x1} + P(X=x_{2})t^{x2} + P(X=x_{3})t^{x3} + … P(X=x_{n})t^{xn}, which can be more concisely written as G_{X}(t) = Σt^{xi}P(X=x_{i}) (known as the closed form of this PGF).

Notice that G_{X}(t) is also the same as the expectation function for t^{x}, so E(t^{x}) = G_{X}(t) = Σt^{xi}P(X=x_{i}).

The t we are using is effectively a dummy variable that is useful for us.

**Worked Example. PGF**

Consider the following probability distribution:

x | 0 | 1 | 2 | 3 | 4 | 5 | 6 |

P(X = x) | 0.1 | 0.2 | 0.3 | 0.15 | 0.1 | 0.1 | 0.05 |

Write down the PGF for the random variable X.

**Worked Example. PGF 2**

Consider the following probability distribution:

x | 2 | 4 | 5 | 10 |

P(X = x) | 0.1 | 0.2 | 0.3 | 0.4 |

Write down the PGF for the random variable X.

You probably noticed in the above examples that the probabilities are simply the coefficients of the t terms. Hence these coefficients sum to 1. So G_{X}(1) = 1.

If we differentiate the PGF with respect to t, we will cause each term to be multiplied by the value x_{i}, giving G’_{X}(t) = Σx_{i}(t)^{xi-1}P(X=x_{i}).

So G’_{X}(1) = Σx_{i}(1)^{xi-1}P(X=x_{i}) = Σx_{i}P(X=x_{i}), which is the same as E(X). Hence G’_{X}(1) = E(X).

**Worked Example. PGF 3**

Let X be a discrete random variable, as showing in the probability distribution given by:

x | 1 | 2 | 3 | 4 | 5 |

P(X = x) | 0.2 | 0.2 | 0.2 | 0.2 | 0.2 |

Find the probability generating function for X.

**Standard discrete distributions**

**Discrete uniform distribution:** If X is a discrete random variable, with a uniform distribution, that is P(X = x_{i}) = 1/n for i = 1,2, .., n, then

**Binomial distribution:** Let X ~ Bi(n,p). Then

**Geometric distribution:** Let X ~ Geo(p). Then

**Poisson distribution:** Let X ~ Po(𝞴). Then

**Worked Example Binary Distribution**

Let x ~ Bi(5,0.2). Find the probability generating function for X.

**Worked Example Geometric Distribution**

Let x ~ Geo(1/5). Find the probability generating function for X.

**Exercise 1**

**Answers to Exercise 1**

Worked solutions to Exercise 1

**Using the Probability Generating Function to calculate Mean and Variance**

We have seen above that G_{X}(t) = Σt^{x}P(X=x).

And from this it follows that G’_{X}(t) = Σxt^{x-1}P(X=x) and

G”_{X}(t) = Σx(x-1)t^{x-2}P(X=x) = Σ(x^{2}-x)t^{x-2}P(X=x) = Σx^{2}t^{x-2}P(X=x) – Σxt^{x-2}P(X=x)

If we evaluate at t=1, we get G”_{X}(1) = Σx^{2}P(X=x) – ΣxtP(X=x)

So G”_{X}(1) = E(X^{2}) – E(X) which we rearrange to give: E(X^{2}) = G”_{X}(1) + G’_{X}(1) ( as we showed above that G’_{X}(1) = E(X).

Hence, **Var(X) =** E(X^{2}) – [E(X)]^{2}= **G” _{X}(1) + G’_{X}(1) – [G’_{X}(1)]^{2}**

**Worked Example. Mean and Variance using PGF**

A bag contains 5 red balls and 3 green balls. The balls are taken out one at a time, the colour is noted, and then it is replaced. Let X be the number of times that a ball is removed until a green ball is chosen.

(a) State the PGF of X.

(b) Calculate the mean and variance of X.

**Worked Example. Mean and Variance using PGF** **(Poisson)**

Prove that for X ~ Po(𝞴):

(a) E(X) = 𝞴

(b) Var(X) = 𝞴

**Worked Example PGF**

A discrete random variable has the following probability distribution:

x | 0 | 1 | 2 |

P(X = x) | a | b | c |

The mean is 2/3 and the variance is 5/9. Find a, b and c.

**Exercise 2**

**Answers to Exercise 2**

Worked Solutions to Exercise 2

**Sum of Independent Random Variables**

Statistics 2 looked at the situation where we have random variable X with Normal distribution N(μ_{1},𝞼_{1}^{2}) and random variable Y with Normal distribution N(μ_{2},𝞼_{2}^{2}) and we are interested in the distribution of X+Y, which is N(μ_{1} + μ_{2}, 𝞼_{1}^{2} + 𝞼_{2}^{2}). The same applies for the Poisson distribution.

We are now interested in finding the PGF of X+Y for independent random variables X and Y. We will consider discrete RVs.

Let us consider X, which has the following probability distribution:

x | 0 | 1 | 2 |

P(X=x) | p_{0} | p_{1} | p_{2} |

Let us also consider Y, which has the following probability distribution:

x | 0 | 1 | 2 |

P(X=x) | q_{0} | q_{1} | q_{2} |

We can hence see that the distribution of X+Y is:

x+y | P(X+Y = x+y) |

0 | P (X = 0 ∩ Y = 0) |

1 | P (X = 0 ∩ Y = 1) + P (X = 1 ∩ Y = 0) |

2 | P (X = 0 ∩ Y = 2) + P (X = 1 ∩ Y = 1) + P (X = 2 ∩ Y = 0) |

3 | P (X = 1 ∩ Y = 2) + P (X = 2 ∩ Y = 1) |

4 | P (X = 2 ∩ Y = 2) |

If X and Y are independent, then P(X=x_{i} ∩ Y=y_{j}) = P(X=x_{i}) x P(Y=y_{j}) = p_{i}q_{j}, so we can simplify the table as:

x+y | P(X+Y = x+y) |

0 | p_{0}q_{0} |

1 | p_{1}q_{0} + p_{0}q_{1} |

2 | p_{2}q_{0} + p_{1}q_{1} + p_{0}q_{2} |

3 | p_{2}q_{1} + p_{1}q_{2} |

4 | p_{2}q_{2} |

So we have the PGF G_{X+Y}(t) = p_{0}q_{0} +(p_{1}q_{0} + p_{0}q_{1})t + (p_{2}q_{0} + p_{1}q_{1} + p_{0}q_{2})t^{2} + (p_{2}q_{1} + p_{1}q_{2})t^{3} +p_{2}q_{2}t^{4} which can be rewritten as G_{X+Y}(t) = (p_{0} + p_{1}t + p_{2}t^{2})(q_{0} + q_{1}t + q_{2}t^{2}), which are the PGFs of X and Y, so in fact **G _{X+Y}(t) = G_{X}(t) x G_{Y}(t)**, which we call the

**convolution theorem**.

**Worked Example. Convolution Theorem**

The discrete random variables X and Y have the following probability distributions:

x | 1 | 2 | 3 |

P(X = x) | 1/4 | 1/4 | 1/2 |

x | 2 | 4 | 6 |

P(Y = y) | 1/3 | 1/3 | 1/3 |

Assuming that X and Y are independent,

(a) Find the PGF of X + Y

(b) Write down the probability distribution of X + Y

(c) Show that E(X+Y) = E(X) + E(Y) and Var(X+Y) = Var(X) + Var(Y).

**The PGF of a function of a random variable**

Let Y = aX + b, where X has the PGF G_{X}(t). We use the fact that G_{X}(t) = E(t^{X}).

So G_{Y}(t) = E(t^{Y})

= E(t^{aX+b})

= E(t^{aX}t^{b})

= t^{b} x E(t^{aX})

= t^{b} x E[(t^{a})^{X}]

= t^{b} x G_{X}(t^{a})

So, G_{aX+b}(t) = t^{b}G_{X}(t^{a}).

Following from this result, we have E(aX+b) = aE(X) + b and Var(aX+b) = a^{2}Var(X)

**Worked Example. Function of Random Variable**

A discrete random variable X has the probability distribution:

x | 1 | 2 | 3 | 4 | 5 |

P(X = x) | 1/9 | 2/9 | 3/9 | 2/9 | 1/9 |

(a) Find G_{X}(t), the PGF of X

(b) Given that Y = 4 – 7X, find G_{Y}(t), the PGF of Y.

**Exercise 3**

**Answers to Exercise 3**

**Worked Solutions to Exercise 3**

**Three or more Random Variables**

The results we have found generalise to deal with linear combinations of more than two random variables..

So, for independent random variables X_{i} with corresponding PGFs G_{Xi}(t),

G_{X1+…+Xn}(t) = G_{X1}(t) x G_{X2}(t) x … x G_{Xn}(t).

If the n discrete random variables all have the same PGF, the formula reduces to G_{X1+…+Xn}(t) = [G_{X}(t)]^{n}.

Also, G_{aX1 + bX2} = G_{X1}(t^{a}) x G_{X2}(t^{b}).

**Worked Example. Six Random Variables**

Find the PGF for the total number of 8s when a faire 8-sided dice is rolled six times.

**Exercise 4**

**Answers to Exercise 4**

Worked Solutions to Exercise 4

**End of PGF Chapter Mixed Questions**

**Answers to End of PGF Chapter Mixed Questions**