Jointly distributed random variables: formula sheet

Key formulas

Joint pmf conditions

p(x, y) = P(X = x, Y = y); p(x, y) ≥ 0; Σ Σ p(x, y) = 1

The double sum runs over all possible pairs. Use this to find an unknown constant.

Joint pdf conditions

f(x, y) ≥ 0; ∫∫ f(x, y) dx dy = 1 over the whole plane

Integrate over the support only, where f is non-zero. Use this to find an unknown constant.

Probability over a region (discrete)

P((X, Y) ∈ A) = Σ p(x, y) over pairs (x, y) in A

List the pairs in A, then add.

Probability over a region (continuous)

P((X, Y) ∈ A) = ∫∫_A f(x, y) dx dy

The limits must describe A intersected with the support.

Joint cdf

F(x, y) = P(X ≤ x, Y ≤ y)

Defined for discrete and continuous variables.

Joint cdf from joint pdf

F(x, y) = ∫ from -∞ to x ∫ from -∞ to y f(u, v) dv du

Only the part of the range inside the support contributes.

Joint pdf from joint cdf

f(x, y) = ∂²F(x, y) ÷ ∂x∂y

Holds where the derivative exists. For discrete variables, differences of F give p instead.

Rectangle probability from the cdf

P(a < X ≤ b, c < Y ≤ d) = F(b, d) − F(a, d) − F(b, c) + F(a, c)

Add back F(a, c) because it is subtracted twice.

Marginal pmf (discrete)

pX(x) = Σy p(x, y); pY(y) = Σx p(x, y)

Sum over all values of the other variable. The marginal probabilities must add to 1.

Marginal pdf (continuous)

fX(x) = ∫ f(x, y) dy; fY(y) = ∫ f(x, y) dx

Integrate over the whole range of the other variable for that fixed value. Limits may depend on x or y when the support is not a rectangle.

Independence condition

f(x, y) = fX(x) × fY(y) for all (x, y)

For discrete variables use p(x, y) = pX(x) × pY(y). A single pair that fails proves the variables are not independent.

Joint CDF form of independence

F(x, y) = FX(x) × FY(y) for all (x, y)

Equivalent to the pdf condition. Useful when the CDF is given.

Conditional pdf

f(y | x) = f(x, y) ÷ fX(x), for fX(x) > 0

If X and Y are independent, f(y | x) = fY(y).

Expectation under independence

E[g(X) h(Y)] = E[g(X)] × E[h(Y)]

Holds when X and Y are independent and the expectations exist. Gives Cov(X, Y) = 0.

Covariance

Cov(X, Y) = E[XY] − E[X] E[Y]

Zero covariance does not imply independence.

Conditional pmf

P(X = x | Y = y) = P(X = x, Y = y) ÷ P(Y = y)

Needs P(Y = y) > 0.

Conditional pdf

f(x | y) = f(x, y) ÷ f_Y(y)

Needs f_Y(y) > 0. Integrates to 1 over x for each fixed y.

Marginal from joint

f_Y(y) = ∫ f(x, y) dx (or Σ over x for a pmf)

Integrate over the full range of x for that y.

Conditional expectation

E[X | Y = y] = ∫ x f(x | y) dx (or Σ x P(X = x | Y = y))

A function of y. The same rule gives E[g(X) | Y = y] using g(x).

Conditional variance

Var(X | Y = y) = E[X² | Y = y] − (E[X | Y = y])²

Use the conditional second moment, not the unconditional one.

Tower law (law of total expectation)

E[X] = E[E[X | Y]]

The outer expectation is over Y.

Variance decomposition (law of total variance)

Var(X) = E[Var(X | Y)] + Var(E[X | Y])

Within-group variance plus between-group variance.

Independence

f(x | y) = f_X(x) for all y, so E[X | Y] = E[X]

Independence implies this. The converse for the mean alone does not hold.

Random sum

S = X₁ + … + X_N, with N independent of the Xᵢ (iid, mean μ, variance σ²): E[S] = μE[N], Var(S) = σ²E[N] + μ²Var(N)

Follows directly from the tower law and variance decomposition.

Expectation of a function

E[g(X,Y)] = ∫∫ g(x,y) f(x,y) dy dx (continuous); Σ Σ g(x,y) P(X=x, Y=y) (discrete)

Integrate over the full support of the joint pdf. Watch for limits that depend on the other variable.

Covariance definition

Cov(X,Y) = E[(X − μX)(Y − μY)]

Use this to understand the idea. It is rarely the fastest way to calculate.

Covariance computing formula

Cov(X,Y) = E[XY] − E[X]E[Y]

Use this in most calculations.

Correlation coefficient

ρ = Corr(X,Y) = Cov(X,Y) ÷ √(Var(X) Var(Y))

Defined only when both variances are positive and finite. −1 ≤ ρ ≤ 1.

Variance of a sum

Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y)

Holds for any X and Y with finite variances, dependent or not.

Variance of a linear combination

Var(aX + bY) = a²Var(X) + b²Var(Y) + 2ab Cov(X,Y)

For a difference, b is negative, so the covariance term changes sign.

Covariance with constants

Cov(aX + b, cY + d) = ac Cov(X,Y)

Adding a constant does not change covariance. Correlation changes sign only if ac is negative.

Independence

If X and Y are independent, Cov(X,Y) = 0 and E[XY] = E[X]E[Y]

The reverse is not true. Zero covariance does not imply independence.

Covariance of sums

Cov(X + Y, Z) = Cov(X,Z) + Cov(Y,Z); Cov(X,X) = Var(X)

Covariance is bilinear and symmetric.

Change of variables (one variable)

If Y = g(X), g strictly monotonic with inverse x = h(y): f_Y(y) = f_X(h(y)) × |h′(y)|

Apply on the range of y that corresponds to the range of x.

Change of variables (two variables)

f_{U,V}(u, v) = f_{X,Y}(x(u,v), y(u,v)) × |J|, where J = ∂(x,y)/∂(u,v) = (∂x/∂u)(∂y/∂v) − (∂x/∂v)(∂y/∂u)

J is the Jacobian of the inverse transformation. Take the absolute value. The transformation must be one-to-one on the support.

Convolution (continuous)

f_{X+Y}(z) = ∫ f_X(x) f_Y(z − x) dx

Requires X and Y independent. Limits are set by where both densities are non-zero.

Convolution (discrete)

P(X + Y = z) = Σ P(X = x) P(Y = z − x)

Requires independence. Sum over all x giving valid values.

MGF of a sum

M_{X+Y}(t) = M_X(t) × M_Y(t)

Requires independence. Extends to n variables as a product of n MGFs.

MGF of a linear function

M_{aX+b}(t) = e^{bt} M_X(at)

Holds for any random variable whose MGF exists.

Maximum of independent variables

F_max(m) = F_X(m) × F_Y(m); for n i.i.d. variables, F_max(m) = [F(m)]ⁿ

Max ≤ m means every variable ≤ m.

Minimum of independent variables

P(min > m) = [1 − F_X(m)] × [1 − F_Y(m)]; for n i.i.d., F_min(m) = 1 − [1 − F(m)]ⁿ

Min > m means every variable > m.

Mean and variance of a sum

E(X + Y) = E(X) + E(Y); Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)

The mean rule always holds. The covariance term is zero when X and Y are independent.

Quick revision

• Joint pmf: p(x, y) ≥ 0 and Σ Σ p(x, y) = 1. Joint pdf: f(x, y) ≥ 0 and the double integral over the support equals 1.
• Marginal of X: sum p(x, y) over y, or integrate f(x, y) over y, across the full range of y for that x.
• Conditional: f(x | y) = f(x, y) ÷ f_Y(y), valid only where f_Y(y) > 0.
• X and Y are independent if and only if f(x, y) = f_X(x) f_Y(y) for all x, y. The support must also be a rectangle.
• E[X] = E[E[X | Y]] and Var(X) = E[Var(X | Y)] + Var(E[X | Y]).
• Cov(X, Y) = E[XY] − E[X]E[Y].
• Correlation ρ = Cov(X, Y) ÷ (σ_X σ_Y), and it always lies between −1 and 1.
• Independent implies Cov = 0, but Cov = 0 does not imply independent.
• Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y). For independent variables the covariance term is zero.
• Sum of independent variables: the MGF of X + Y is the product of the MGFs.
• Sum of independent Poisson variables with means λ₁ and λ₂ is Poisson with mean λ₁ + λ₂.
• Change of variable: multiply by the absolute value of the Jacobian, and recompute the new support.

Common mistakes

• Integrating over the wrong region, such as the whole unit square when the event is X < Y. Fix: Always sketch the support and shade the event. Write limits only after reading them off the sketch.
• Using constant limits for both variables when the support is a triangle, such as 0 < x < y < 1. Fix: The inner integral's limits must depend on the outer variable. The outer limits must be constants.
• Ignoring the support when testing independence. Fix: Always check the region. If the limits of one variable depend on the other, the variables are not independent. Here fX(x) = 2(1 − x) and fY(y) = 2y, and their product 4y(1 − x) is not 2.
• Using the wrong limits when integrating out a variable. Fix: Sketch the region. For fixed x, read off where y starts and ends. Write the limits before you integrate.
• Dividing by the wrong marginal, or not dividing at all, so the conditional density does not integrate to 1. Fix: To condition on Y = y, divide by f_Y(y). Always check that the result integrates to 1 over x.
• Using wrong limits when finding the marginal or the conditional mean. Fix: Sketch the region. For f_Y(y), integrate x over its range for that fixed y. Use the same range for E[X | Y = y].
• Writing E[XY] = E[X]E[Y] without checking independence Fix: Use it only when you can show independence. Otherwise compute E[XY] from the joint distribution.
• Concluding that zero covariance means independence Fix: Covariance measures only linear association. Independence needs the joint distribution to factorise.
• Using convolution or the product of MGFs when the variables are not independent. Fix: State independence explicitly before using the formula. If it is not given, use the joint pdf and the CDF or change of variables method.
• Forgetting the absolute value of the Jacobian, or using the Jacobian of the forward map instead of the inverse. Fix: Write x and y as functions of u and v first, then differentiate. Always write |J|. If you used the forward map, divide by it instead.

Exam tips

• Draw the support and the event region in every continuous question, even in MCQs. Show the sketch or the limits in written answers, since method marks depend on correct limits.
• Find any unknown constant first and state its value. A wrong k carries through every later part, so check it by recomputing the total.
• Write the notation exactly: f(x, y) for the pdf, p(x, y) for the pmf, F(x, y) = P(X ≤ x, Y ≤ y) for the cdf. State the support each time.
• Use a quick sanity check: a probability must lie between 0 and 1, and P(event) + P(complement) = 1. For constant pdfs, compare areas.
• If a part asks for a marginal or conditional distribution next, keep your joint pdf and support clearly written. Those parts build directly on it.
• Always give the range of the marginal along with its formula. Marks are usually allocated to it.
• Check the support before any algebra. A non-rectangular region settles the independence question straight away.
• In MCQs, a quick test is to look for a term like x + y in the joint pdf. It cannot factorise, so the variables are not independent.