A data scientist uses LIME to explain why a loan application was denied: 'income' contributes –0.3 and 'debt' contributes –0.4. She reruns LIME on the same prediction and gets slightly different numbers. What explains this?
ALIME is broken; explanations for the same prediction must always be identical
BLIME generates perturbed samples randomly, so different sampling runs produce slightly different local linear fits
CSHAP was accidentally used instead; SHAP produces variable results between runs
DThe model changed between runs, producing different predictions
LIME works by randomly perturbing the input, feeding perturbations through the black-box model, and fitting a local linear model weighted by proximity. Because perturbations are sampled randomly, different runs yield different datasets and thus slightly different linear fits — a known limitation. SHAP, by contrast, has a unique deterministic solution grounded in Shapley values, making its attributions consistent across runs.
Question 2 True / False
A credit scoring model uses 50 features. SHAP's feature attributions for a given prediction will always sum to the model's prediction minus the average prediction across all training examples.
TTrue
FFalse
Answer: True
This is SHAP's 'efficiency' axiom from Shapley value theory: the sum of all SHAP values for a prediction equals the difference between that specific prediction and the model's baseline (typically the average prediction). This means the explanation accounts for the full prediction with no unexplained residual — a theoretical guarantee that LIME does not provide.
Question 3 True / False
LIME and SHAP both explain individual model predictions, so they can be used interchangeably for any explanation task.
TTrue
FFalse
Answer: False
While both are model-agnostic local explanation methods, they differ importantly. LIME can vary between runs (different perturbation samples) and has no global consistency guarantee. SHAP provides theoretically grounded, consistent attributions that can be aggregated across predictions for global summaries (e.g., SHAP summary plots). For high-stakes or global feature importance analysis, SHAP is preferable; for quick exploratory explanations, LIME may suffice.
Question 4 Short Answer
What is the key idea behind Shapley values, and why does it make SHAP's attribution more principled than a model's internal feature weights?
Think about your answer, then reveal below.
Model answer: Shapley values compute each feature's average marginal contribution across all possible subsets of features — not just when all features are present simultaneously. This correctly handles feature interactions and redundancies. A model's internal weight measures a feature's contribution given all other features, which can be unstable or misleading when features are correlated. Shapley values distribute credit fairly among all features by averaging over every possible ordering of feature inclusion.
If two features are highly correlated, internal weights can split arbitrarily between them. Shapley values solve this by considering every possible coalition: how much does feature A contribute when only it is present? When it and B are present? All three? Averaging over these coalitions gives stable, fair attributions even with correlated features.
Question 5 Multiple Choice
TreeSHAP is preferred over KernelSHAP for gradient-boosted tree models primarily because:
ATreeSHAP uses LIME's local approximation approach, which is faster for tree-structured models
BTreeSHAP exploits the tree structure to compute exact Shapley values in polynomial time, while KernelSHAP requires sampling and is approximate
CKernelSHAP cannot handle categorical features, which tree models commonly use
DTreeSHAP produces higher attribution magnitudes, making features appear more important
Exact Shapley values require evaluating the model on exponentially many feature subsets — infeasible for high-dimensional inputs. KernelSHAP approximates this by sampling. TreeSHAP exploits the tree data structure to compute exact Shapley values in polynomial time (O(TLD²) for T trees, L leaves, D depth). This makes TreeSHAP both faster and exact for tree-based models like XGBoost or random forests.