Questions: Multilevel Cache Design and Coordination

Question 1

A multicore processor uses an inclusive L3 cache. Core A wants to modify a cache line that Core B might have cached. Why does the inclusive design simplify this coherence check?

• A. Inclusive caches use hardware locks that serialize all cache accesses across cores
• B. Because every L1 and L2 cache line is guaranteed to exist in L3, the coherence protocol only needs to probe L3 — if the line isn't there, it isn't anywhere
• C. Inclusive caches are smaller and faster to search than exclusive caches
• D. The shared L3 directly controls all L1 and L2 caches, so it can invalidate them without probing

Answer: (B)

The inclusion property guarantees that L3 is a superset of all data in any L1 or L2 on any core. A coherence check can therefore stop at L3: a cache line not in L3 cannot be in any private cache. Without this guarantee (as in exclusive or NINE caches), the protocol must potentially search across all levels, significantly increasing coherence complexity and latency.

Question 2

Which workload scenario best illustrates the performance advantage of an exclusive cache hierarchy over an inclusive one?

• A. Multiple cores frequently sharing the same hot data, requiring rapid cross-core coherence checks
• B. Each core accessing a distinct, large private dataset where L2 misses rarely repeat, so avoiding duplicated data across levels maximizes total effective capacity
• C. A workload requiring very frequent L1 hits, where L3 behavior is largely irrelevant
• D. A single-threaded workload with a working set that fits entirely within L1

Answer: (B)

Exclusive caches eliminate data duplication: each line exists at exactly one level, so the sum of all cache sizes equals the effective total capacity. For workloads with large, private per-core datasets where duplicating data across levels wastes capacity, exclusive hierarchies significantly outperform inclusive ones. The tradeoff is more complex coherence protocols, which is why inclusive caches with their simpler probing are preferred when cross-core sharing is common.

Question 3

In an inclusive cache hierarchy, every cache line present in an L1 cache is also guaranteed to be present in L2 and L3.

• T. True
• F. False

Answer: True

Inclusive policy means each lower level is a superset of all higher levels combined. L3 contains everything in L2, which contains everything in L1. This redundancy simplifies coherence at the cost of effective capacity — L3 area is partially 'wasted' holding copies of L1 and L2 data rather than additional unique data.

Question 4

A program whose entire working set fits within the L2 cache will run significantly faster on a processor with a larger L3 cache.

• T. True
• F. False

Answer: False

If the working set fits in L2, memory accesses are satisfied without ever reaching L3. A larger L3 provides no latency benefit because L3 is never consulted. L3 only helps when L2 misses — which don't occur here. This illustrates why understanding your workload's working set size relative to each cache level is essential for predicting where cache size improvements will actually improve performance.

Question 5

Explain why an exclusive cache hierarchy achieves greater total effective capacity than an inclusive one with the same physical cache sizes, and what tradeoff this introduces.

Model answer: In an inclusive hierarchy, every L1 and L2 cache line is duplicated in L3, so L3's capacity is partially consumed by holding redundant copies rather than additional unique data — the effective unique data capacity is bounded by L3 alone. In an exclusive hierarchy, each cache line resides at exactly one level, so total effective capacity is the sum of all levels (L1 + L2 + L3). The tradeoff is coherence complexity: the inclusive property allows coherence checks to stop at L3 (if not there, not anywhere), whereas exclusive hierarchies require probing across levels to locate a line, making the coherence protocol more complex and potentially slower.

Real processors often use non-inclusive, non-exclusive (NINE) policies as a pragmatic middle ground — lines are not guaranteed to be duplicated, but are not guaranteed to be unique either. This gives more flexibility in managing capacity while keeping coherence protocols tractable.