Solved Problems on Law of Total Expectation & Variance (Step-by-Step)

📚 Understanding the Law of Total Expectation

The Law of Total Expectation, also known as the tower rule or iterative expectation, provides a way to calculate the expected value of a random variable when you know the conditional expectation of that variable given another random variable. It's super useful in probability and statistics for breaking down complex problems into simpler, conditional components.

• 🔍 Definition: The expected value of a random variable $X$ can be calculated by averaging the conditional expectations of $X$ given another random variable $Y$, weighted by the probability distribution of $Y$.
• 💡 Formula: $E[X] = E[E[X|Y]]$
• 📝 Intuition: Think of it as averaging the averages. You first compute the average of $X$ for each possible value of $Y$, and then average those averages using the distribution of $Y$.

📜 A Brief History

The Law of Total Expectation, while not attributed to a single individual, emerged from the development of probability theory in the 20th century. It's a natural extension of conditional probability and expectation concepts, solidifying its place as a fundamental tool in stochastic processes and statistical inference.

🔑 Key Principles

• 🧮 Conditional Expectation: Understanding what $E[X|Y=y]$ means is crucial. It's the expected value of $X$ given that $Y$ takes on a specific value $y$.
• 🎲 Random Variables: Both $X$ and $Y$ must be random variables defined on the same probability space.
• ⚖️ Averaging: The outer expectation $E[E[X|Y]]$ averages the conditional expectations over all possible values of $Y$.

🏢 Real-World Example: Sales Performance

Imagine you manage a sales team. The number of sales ($X$) depends on whether or not there's a promotional campaign ($Y$). Let $Y = 1$ if there's a campaign and $Y = 0$ if there isn't. You know:

• 📊 If there's a campaign (Y=1): The expected number of sales per salesperson is $E[X|Y=1] = 20$.
• 📉 If there's no campaign (Y=0): The expected number of sales per salesperson is $E[X|Y=0] = 10$.
• 📅 Probability of a campaign: The probability of having a campaign is $P(Y=1) = 0.4$, so $P(Y=0) = 0.6$.

To find the overall expected number of sales per salesperson, you use the Law of Total Expectation:

$E[X] = E[E[X|Y]] = E[X|Y=1]P(Y=1) + E[X|Y=0]P(Y=0) = 20 * 0.4 + 10 * 0.6 = 8 + 6 = 14$

So, the expected number of sales per salesperson is 14.

🧪 Solved Problems (Step-by-Step)

Problem 1: Coin Toss and Expected Value

A coin is tossed. If it lands heads, you draw a number from a uniform distribution between 0 and 1. If it lands tails, you draw a number from a uniform distribution between 1 and 2. What is the expected value of the number you draw?

• 🪙 Define Random Variables: Let $X$ be the number you draw, and let $Y$ be 1 if the coin lands heads and 0 if it lands tails.
• 🎯 Given Information:
  • $P(Y=1) = P(Y=0) = 0.5$
  • $E[X|Y=1] = 0.5$ (expected value of a uniform distribution between 0 and 1)
  • $E[X|Y=0] = 1.5$ (expected value of a uniform distribution between 1 and 2)
• ✅ Apply the Law: $E[X] = E[E[X|Y]] = E[X|Y=1]P(Y=1) + E[X|Y=0]P(Y=0) = 0.5 * 0.5 + 1.5 * 0.5 = 0.25 + 0.75 = 1$
• 🎉 Answer: The expected value of the number you draw is 1.

Problem 2: Two Dice and Conditional Expectation

You roll two fair six-sided dice. Let $X$ be the sum of the two dice and $Y$ be the value of the first die. What is $E[X|Y]$?

• 🎲 Define Random Variables: $X$ is the sum, $Y$ is the first die.
• 🎯 Understand Conditional Expectation: $E[X|Y=y]$ is the expected sum given the first die is $y$.
• ✅ Calculate: If the first die is $y$, then the expected value of the second die is 3.5. Therefore, $E[X|Y=y] = y + 3.5$.
• 🎉 Answer: $E[X|Y] = Y + 3.5$

Problem 3: Urn Problem

An urn contains 5 red balls and 5 blue balls. You randomly draw two balls without replacement. Let $X$ be the number of red balls drawn. What is $E[X]$?

• 🔴 Define Random Variables: $X$ = number of red balls drawn. Let $Y$ be the number of red balls drawn on the first draw (1 if red, 0 if blue).
• 🎯 Conditional Probabilities:
  • $P(Y=1) = 5/10 = 0.5$
  • $P(Y=0) = 5/10 = 0.5$
• 🧮 Conditional Expectations:
  • $E[X|Y=1] = 1 + E[ ext{red balls on 2nd draw} | ext{1st ball red}] = 1 + 4/9$
  • $E[X|Y=0] = 0 + E[ ext{red balls on 2nd draw} | ext{1st ball blue}] = 0 + 5/9$
• ✅ Apply the Law: $E[X] = E[E[X|Y]] = (1 + 4/9) * 0.5 + (5/9) * 0.5 = (13/9) * 0.5 + (5/9) * 0.5 = (18/9) * 0.5 = 2 * 0.5 = 1$
• 🎉 Answer: The expected number of red balls drawn is 1.

📚 Understanding the Law of Total Variance

The Law of Total Variance expresses the variance of a random variable in terms of the variance of its conditional expectation and the expected value of its conditional variance. It's another powerful tool for breaking down variance calculations.

• 🔍 Definition: The variance of a random variable $X$ can be calculated by considering the conditional variance of $X$ given another random variable $Y$, and the variance of the conditional expectation of $X$ given $Y$.
• 💡 Formula: $Var(X) = E[Var(X|Y)] + Var(E[X|Y])$
• 📝 Intuition: The total variance consists of two parts: the average variance within each group defined by $Y$, and the variance between the means of those groups.

🔑 Key Principles

• 🧮 Conditional Variance: $Var(X|Y=y)$ measures the variability of $X$ given that $Y$ takes on a specific value $y$.
• 🎲 Random Variables: As with the Law of Total Expectation, both $X$ and $Y$ must be random variables defined on the same probability space.
• ➕ Decomposition: The Law of Total Variance decomposes the total variance into within-group variance and between-group variance.

🏢 Real-World Example: Exam Scores

Suppose exam scores ($X$) depend on whether students studied ($Y=1$) or didn't ($Y=0$).

• 📊 If they studied (Y=1): $E[X|Y=1] = 80$, $Var(X|Y=1) = 100$
• 📉 If they didn't (Y=0): $E[X|Y=0] = 60$, $Var(X|Y=0) = 225$
• 📅 Probability of studying: $P(Y=1) = 0.6$, so $P(Y=0) = 0.4$.

First, find $E[X] = E[E[X|Y]] = 80*0.6 + 60*0.4 = 48 + 24 = 72$. Then:

• 🧮 $E[Var(X|Y)] = Var(X|Y=1)P(Y=1) + Var(X|Y=0)P(Y=0) = 100 * 0.6 + 225 * 0.4 = 60 + 90 = 150$
• 🧮 $Var(E[X|Y]) = E[(E[X|Y])^2] - (E[E[X|Y]])^2 = E[(E[X|Y])^2] - (E[X])^2$. We know $E[X] = 72$. $E[(E[X|Y])^2] = 80^2 * 0.6 + 60^2 * 0.4 = 6400 * 0.6 + 3600 * 0.4 = 3840 + 1440 = 5280$. So, $Var(E[X|Y]) = 5280 - 72^2 = 5280 - 5184 = 96$.

Finally, $Var(X) = E[Var(X|Y)] + Var(E[X|Y]) = 150 + 96 = 246$

🧪 Solved Problems (Step-by-Step)

Problem 4: Coin Toss and Variance

A coin is tossed. If it lands heads, you draw a number from a uniform distribution between 0 and 1. If it lands tails, you draw a number from a uniform distribution between 1 and 2. What is the variance of the number you draw?

• 🪙 Define Random Variables: Let $X$ be the number you draw, and let $Y$ be 1 if the coin lands heads and 0 if it lands tails.
• 🎯 Given Information:
  • $P(Y=1) = P(Y=0) = 0.5$
  • $E[X|Y=1] = 0.5$, $Var(X|Y=1) = 1/12$ (variance of a uniform distribution between 0 and 1)
  • $E[X|Y=0] = 1.5$, $Var(X|Y=0) = 1/12$ (variance of a uniform distribution between 1 and 2)
• ✅ Apply the Law:
  • $E[Var(X|Y)] = Var(X|Y=1)P(Y=1) + Var(X|Y=0)P(Y=0) = (1/12) * 0.5 + (1/12) * 0.5 = 1/12$
  • $Var(E[X|Y]) = E[(E[X|Y])^2] - (E[X])^2 = (0.5^2 * 0.5 + 1.5^2 * 0.5) - 1^2 = (0.125 + 1.125) - 1 = 1.25 - 1 = 0.25$
  • $Var(X) = E[Var(X|Y)] + Var(E[X|Y]) = 1/12 + 0.25 = 1/12 + 3/12 = 4/12 = 1/3$
• 🎉 Answer: The variance of the number you draw is 1/3.

Problem 5: Dice Roll and Variance

Roll a fair six-sided die. Let $X$ be the outcome. Now, roll a second die $Y$ times. Let $S$ be the sum of the outcomes of these $Y$ rolls. What is $Var(S)$?

• 🎲 Define Random Variables: $Y$ is the number of rolls, and $S$ is the sum of the rolls.
• 🎯 Known Information: $E[Y] = 3.5$, $Var(Y) = 35/12$. For a single die roll, the mean is 3.5 and the variance is 35/12.
• ✅ Conditional Expectation and Variance: $E[S|Y] = 3.5Y$ and $Var(S|Y) = (35/12)Y$
• 🧮 Apply the Law of Total Variance:
  • $E[Var(S|Y)] = E[(35/12)Y] = (35/12)E[Y] = (35/12)(3.5) = 122.5/12$
  • $Var(E[S|Y]) = Var(3.5Y) = 3.5^2 Var(Y) = 12.25 * (35/12) = 428.75/12$
  • $Var(S) = E[Var(S|Y)] + Var(E[S|Y]) = 122.5/12 + 428.75/12 = 551.25/12 = 45.9375$
• 🎉 Answer: $Var(S) = 45.9375$

Problem 6: Random Number of Trials

Suppose $X_1, X_2, ...$ are independent and identically distributed (i.i.d.) random variables with mean $\mu$ and variance $\sigma^2$. Let $N$ be a non-negative integer-valued random variable, independent of the $X_i$'s, with mean $E[N] = \lambda$ and variance $Var(N) = \tau^2$. Let $S = \sum_{i=1}^{N} X_i$. Find $Var(S)$.

• 🎯 Known Information: $E[X_i] = \mu$, $Var(X_i) = \sigma^2$, $E[N] = \lambda$, $Var(N) = \tau^2$.
• ✅ Conditional Expectation and Variance: $E[S|N] = N\mu$ and $Var(S|N) = N\sigma^2$
• 🧮 Apply the Law of Total Variance:
  • $E[Var(S|N)] = E[N\sigma^2] = \sigma^2 E[N] = \sigma^2 \lambda$
  • $Var(E[S|N]) = Var(N\mu) = \mu^2 Var(N) = \mu^2 \tau^2$
  • $Var(S) = E[Var(S|N)] + Var(E[S|N]) = \sigma^2 \lambda + \mu^2 \tau^2$
• 🎉 Answer: $Var(S) = \lambda \sigma^2 + \tau^2 \mu^2$

📝 Practice Quiz

Question 1:

A store sells umbrellas. The number of umbrellas sold on a rainy day is $X$, with $E[X] = 50$ and $Var(X) = 25$. On a sunny day, the number of umbrellas sold is $Y$, with $E[Y] = 10$ and $Var(Y) = 9$. The probability of a rainy day is 0.3. What is the expected number of umbrellas sold on a randomly chosen day?

Question 2:

You have two bags. Bag A contains 3 red balls and 2 blue balls. Bag B contains 1 red ball and 4 blue balls. You pick a bag at random (with probability 0.5 for each bag) and draw a ball. What is the probability that the ball is red?

Question 3:

A factory produces light bulbs. The probability that a bulb is defective is 0.05. You select 100 bulbs at random. What is the expected number of defective bulbs?

Question 4:

A random variable $X$ is uniformly distributed between 0 and $Y$, where $Y$ is also a random variable uniformly distributed between 0 and 1. Find $E[X]$.

Question 5:

Let $X$ be a random variable representing the number of cars that pass a certain point on a highway in one hour. Suppose $X$ follows a Poisson distribution with a mean that depends on the weather. If it is sunny, the mean is 10; if it is raining, the mean is 20. If the probability of rain is 0.4, what is the expected number of cars that pass the point in one hour?

Question 6:

Let $X$ be a random variable that represents the amount of time (in minutes) a customer spends at a coffee shop. Suppose that if the customer orders a coffee (event $C$), $X$ follows an exponential distribution with a mean of 10 minutes, and if the customer does not order a coffee (event $C'$), $X$ follows an exponential distribution with a mean of 5 minutes. If $P(C) = 0.6$, find the expected time a customer spends at the coffee shop.

Question 7:

Let $X$ be the outcome of rolling a fair six-sided die. Then, you flip a coin $X$ times. Let $Y$ be the number of heads you get. Find $E[Y]$.

💡 Conclusion

The Law of Total Expectation and the Law of Total Variance are powerful tools in probability and statistics. By understanding and applying these laws, you can tackle complex problems by breaking them down into smaller, more manageable pieces. Keep practicing with different examples to master these concepts! 👍