Conditional Distributions and Conditional Expectation Explained

Understand Conditional Distributions and Conditional Expectation

Key rules to remember

Conditional pmf

P(X = x | Y = y) = P(X = x, Y = y) ÷ P(Y = y)

Needs P(Y = y) > 0.

Conditional pdf

f(x | y) = f(x, y) ÷ f_Y(y)

Needs f_Y(y) > 0. Integrates to 1 over x for each fixed y.

Marginal from joint

f_Y(y) = ∫ f(x, y) dx (or Σ over x for a pmf)

Integrate over the full range of x for that y.

Conditional expectation

E[X | Y = y] = ∫ x f(x | y) dx (or Σ x P(X = x | Y = y))

A function of y. The same rule gives E[g(X) | Y = y] using g(x).

Conditional variance

Var(X | Y = y) = E[X² | Y = y] − (E[X | Y = y])²

Use the conditional second moment, not the unconditional one.

Tower law (law of total expectation)

E[X] = E[E[X | Y]]

The outer expectation is over Y.

Variance decomposition (law of total variance)

Var(X) = E[Var(X | Y)] + Var(E[X | Y])

Within-group variance plus between-group variance.

Independence

f(x | y) = f_X(x) for all y, so E[X | Y] = E[X]

Independence implies this. The converse for the mean alone does not hold.

Random sum

S = X₁ + … + X_N, with N independent of the Xᵢ (iid, mean μ, variance σ²): E[S] = μE[N], Var(S) = σ²E[N] + μ²Var(N)

Follows directly from the tower law and variance decomposition.

How to solve Conditional Distributions and Conditional Expectation questions

1. 1Write down the joint pdf or pmf and its exact support. Note whether the limits of one variable depend on the other.
2. 2Find the marginal of the conditioning variable by integrating (or summing) the joint over the other variable, using the correct limits.
3. 3Divide the joint by that marginal to get the conditional density or pmf. Check it integrates or sums to 1 over the variable of interest.
4. 4Compute E[X | Y = y] from the conditional distribution. Compute E[X² | Y = y] too if you need the variance.
5. 5Get Var(X | Y = y) = E[X² | Y = y] − (E[X | Y = y])².
6. 6If the question asks for an unconditional mean or variance, apply E[X] = E[E[X | Y]] and Var(X) = E[Var(X | Y)] + Var(E[X | Y]), using the distribution of Y.
7. 7Check the result: the answer must lie within the support, and the unconditional mean must lie between the smallest and largest conditional means.

Quickest way: Condition first, then use the tower law

Common mistakes in Conditional Distributions and Conditional Expectation

• Dividing by the wrong marginal, or not dividing at all, so the conditional density does not integrate to 1.

  Students confuse f(x, y), f_X(x) and f_Y(y), or use the joint as if it were conditional.

  Fix: To condition on Y = y, divide by f_Y(y). Always check that the result integrates to 1 over x.

• Using wrong limits when finding the marginal or the conditional mean.

  The support is not a rectangle, for example 0 < x < y < 1, and the dependence between limits is missed.

  Fix: Sketch the region. For f_Y(y), integrate x over its range for that fixed y. Use the same range for E[X | Y = y].

• Writing Var(X) = E[Var(X | Y)] and stopping.

  Students forget the second term of the decomposition.

  Fix: Always include Var(E[X | Y]). It is zero only when E[X | Y] does not depend on Y.

• Treating E[X | Y] as a number.

  Students forget it is a function of y, and a random variable once Y is random.

  Fix: Keep y in the answer. Take the outer expectation or variance over Y only in the last step.

• Using E[X²] where E[X² | Y = y] is needed when computing conditional variance.

  Mixing unconditional and conditional moments.

  Fix: Compute both moments from the same conditional density, then subtract the square of the conditional mean.

• Assuming E[X | Y] = E[X] implies independence.

  Students over-read the independence rule.

  Fix: Independence implies the conditional mean is constant. A constant conditional mean alone does not prove independence.

Worked examples