Questions: Causal Information Theory

Question 1

In a causal graph, two variables X and Z are d-separated given conditioning set S if all paths between them are blocked. What does d-separation imply about information flow?

• A. d-separation means X and Z are always independent
• B. d-separation means X and Z are informationally independent given S in any distribution consistent with the causal graph — formally, I(X;Z|S) = 0 in all Markov distributions over the graph
• C. d-separation guarantees that X does not cause Z
• D. d-separation is only relevant for continuous variables

Answer: (B)

d-separation is a graph-theoretic criterion that implies conditional independence in any probability distribution respecting the causal graph structure. If X and Z are d-separated given S in the graph (all paths between them are blocked by conditioning on S or collider structure), then I(X;Z|S) = 0 for any distribution that respects the graph's conditional independence statements (any Markov distribution). Conversely, if X and Z are not d-separated given S, they may be dependent in some distributions respecting the graph. d-separation is a practical tool: knowing the causal graph structure tells you which conditional independences must hold, which constrains what you can infer.

Question 2

Transfer entropy T(X → Y) measures information flow from time series X to time series Y. It is defined as I(Y_t; X_past | Y_past). Why is conditioning on Y_past necessary to isolate the causal effect of X on Y?

• T. True
• F. False

Answer: True

Y has its own temporal dynamics: Y_t depends on Y_past due to autoregressive structure, trends, or oscillations. To isolate the causal effect of X on Y, we must account for Y's own history. Transfer entropy T(X → Y) = I(Y_t; X_past | Y_past) measures the information in X_past about Y_t that is not already explained by Y_past. If Y_past completely explains Y_t (e.g., Y is purely autoregressive), then T(X → Y) = 0 even if X and Y are correlated (spurious correlation due to common trends). Conversely, if X causally influences Y, then X_past provides information about Y_t beyond Y_past alone. Transfer entropy isolates the causal contribution of X to Y's evolution by conditioning out Y's self-information.

Question 3

Explain why observational data (where variables are passively observed) can yield different conditional independence statements than interventional data (where variables are actively set to specific values). What does this imply for causal discovery?

Model answer: In observational data, variables take their natural values determined by their causal parents and noise. The conditional independences that hold (as given by d-separation) reflect the causal graph structure. However, if a variable X has an unobserved confounder U affecting both X and another variable Y, the observational distribution will show dependence between X and Y even though X does not directly cause Y. Intervention breaks this: if we intervene to set X to a specific value, we sever X's dependence on U, and the conditional distribution of Y given the intervention reflects only X's direct causal effect. Mathematically, P(Y | do(X=x)) differs from P(Y | X=x) when confounders exist. This implies that causal discovery from pure observational data is difficult and requires strong assumptions (no hidden confounders, causal sufficiency). With access to interventional data, causal direction can be determined more reliably. In practice, causal inference from observational data uses sensitivity analyses (how robust are conclusions to potential hidden confounders) and causal assumptions encoded in graphical models.

The distinction between P(Y|X) (observational) and P(Y|do(X)) (interventional) is fundamental. Causal graphs encode assumptions about confounding, and d-separation in the graph determines when observational equivalence holds (different graphs have the same observational distribution). Discovering the true causal structure from observational data alone is an identifiability problem — multiple graphs may be compatible with the data, requiring domain knowledge or additional assumptions to resolve.

Question 4

The causal Markov condition states that each variable is conditionally independent of its non-descendants given its parents in the causal DAG. This condition relates the graph structure to probability distributions. Why is this condition essential for causal inference?

• A. It defines what we mean by a 'causal' graph as opposed to just a dependency graph
• B. It connects causal assumptions (encoded in the graph) to measurable probabilities (conditional independences), allowing data to constrain and test causal hypotheses
• C. It is only a mathematical convenience with no practical importance
• D. It guarantees that causal inference is always identifiable from data

Answer: (B)

The causal Markov condition is the bridge between causal assumptions (the directed acyclic graph structure) and observable probabilities. It says: if the causal graph is correct, then the probability distribution must factor according to d-separation — conditional independences must hold as the graph implies. Conversely, if we observe conditional independences in data, they constrain which causal graphs are plausible. This is why graphical causal models are powerful: they encode causal assumptions, imply testable predictions (conditional independences), and guide causal discovery. Without the Markov condition, causal graphs would be unfalsifiable — graphs could not be tested against data. The Markov condition makes causal assumptions verifiable and refutable.