Questions: Joint Probability Distributions

Question 1

You know the marginal distribution of X and the marginal distribution of Y. Under what condition can you determine the joint probability P(X=x, Y=y) for all pairs?

• A. Always — the joint distribution is just the product of the two marginals
• B. Never — joint distributions contain information that marginals cannot recover under any circumstances
• C. Only when X and Y are independent — then P(X=x, Y=y) = P(X=x)·P(Y=y) for all pairs
• D. When both variables are discrete — continuous variables require additional assumptions

Answer: (C)

Knowing both marginals is NOT enough to determine the joint distribution in general. Many different joint distributions can produce the same marginals — including both independent and highly dependent configurations. Only when X and Y are independent does the joint factor as the product of the marginals, making recovery possible. This is a fundamental asymmetry: you can always get the marginals from the joint (by summing/integrating), but you cannot get the joint from the marginals without additional information.

Question 2

In a joint probability table for discrete X and Y, how do you compute the marginal probability P(X = 2)?

• A. Read the diagonal entry where both X = 2 and Y = 2
• B. Sum all entries in the row where X = 2, across all values of Y
• C. Sum all entries in the column where Y = 2, across all values of X
• D. Divide the total probability mass by the number of rows

Answer: (B)

The marginal PMF of X is found by summing (marginalizing) over all values of Y: P(X = x) = Σ_y P(X = x, Y = y). In the table, this means summing across the entire row for X = 2. Option C gives the marginal of Y at Y = 2, not X at X = 2. This operation collapses the two-variable distribution back to a one-variable distribution for X alone.

Question 3

If two random variables X and Y have marginal distributions identical to those of an independent pair (X', Y'), then X and Y is expected to also be independent.

• T. True
• F. False

Answer: False

Identical marginals do not imply identical joint distributions. Two very different joint distributions — one where X and Y are independent, one where they are strongly dependent — can produce exactly the same marginals. For example, if X and Y each take values {0,1} with equal probability, both an independent joint (where the 2×2 table has equal 0.25 in each cell) and a perfectly correlated joint (where X=Y always) give the same uniform marginals. Independence requires the joint to factor, which is a condition on the joint distribution itself, not recoverable from the marginals alone.

Question 4

The joint distribution of two random variables contains at least as much information as either marginal distribution alone.

• T. True
• F. False

Answer: True

The marginal distributions can always be derived from the joint by summing or integrating out the other variable — they are a coarsening of the joint. But the joint contains additional information the marginals do not: the dependence structure between X and Y. Since you can always recover the marginals from the joint but not vice versa, the joint is strictly more informative (except in the special case of independence, where no additional information is lost).

Question 5

Why is it a mistake to assume two random variables are independent simply because you have no direct evidence of dependence?

Model answer: Independence is a strong structural claim: it requires P(X=x, Y=y) = P(X=x)·P(Y=y) to hold for every single pair (x,y) — or equivalently, that the joint distribution factors exactly as the product of the marginals. Absence of evidence is not evidence of absence. Variables can be strongly dependent in ways that aren't immediately visible from summary statistics, especially in small samples. Assuming independence without verification conflates ignorance with knowledge, potentially leading to incorrect probability calculations, wrong predictions, and flawed conclusions about causal or associative relationships.

In practice, independence must be justified by a physical argument (the variables are generated by unrelated processes), a modeling assumption (made explicit and tested), or a formal statistical test. The prior assumption should always be that variables may be dependent — independence is the special case to be established, not assumed.