A topic in the Open Knowledge Graph — a free, open map of 15,290 topics and the order to learn them in.

Network Information Theory

Research Depth 100 in the knowledge graph I know this Set as goal
5topics build on this
447prerequisites beneath it
See this on the map →
Channel CapacityChannel Coding Theorem+1 moreBroadcast ChannelMultiple Access Channel+1 more
network multi-user distributed relay interference

Core Idea

Network information theory extends Shannon's point-to-point results to multi-user communication networks. When multiple senders and/or receivers share a channel, new phenomena arise that have no single-user analog: cooperation, interference, distributed compression, and capacity regions (sets of achievable rate tuples rather than single numbers). The capacity of the multiple access channel, broadcast channel, and interference channel are central problems. While some multi-user capacity regions are fully characterized (multiple access channel, degraded broadcast channel), others remain open after 50+ years (general interference channel, relay channel), making network information theory one of the most active areas of information-theoretic research.

Explainer

Shannon's original theory considers one sender and one receiver. Real communication systems involve many users: cell phones sharing a base station, devices on a Wi-Fi network, satellites communicating with ground stations. Network information theory asks: what are the fundamental limits when multiple communication sessions share the same physical medium?

The simplest multi-user channels illustrate the key ideas. The multiple access channel (MAC) has K senders transmitting independent messages to one receiver. The capacity region is the set of rate tuples (R_1, ..., R_K) such that for every subset S of users, sum_{i in S} R_i <= I(X_S; Y | X_{Sc}). The receiver can decode all messages using successive interference cancellation: decode one user, subtract their signal, decode the next. The broadcast channel (BC) has one sender transmitting different messages to K receivers. The sender uses superposition coding: layering messages at different power levels so that stronger receivers can decode more layers. The interference channel has K sender-receiver pairs sharing the same medium, where each receiver wants only its own message but hears everyone's signal.

The mathematical challenge is that multi-user problems rarely decompose into independent single-user problems. Interference creates coupling between users. Cooperation (through relays or user coordination) can increase capacity beyond what independent links achieve. Distributed source coding (Slepian-Wolf) shows that correlated sources can be compressed to their joint entropy even when the encoders do not communicate. These phenomena — interference, cooperation, and correlation — are fundamentally multi-user and have no single-user analog.

The state of knowledge is uneven. The MAC capacity region is fully known. The degraded broadcast channel capacity is known (superposition coding). The Gaussian interference channel capacity is known in certain regimes (strong interference, very weak interference) but not in general. The relay channel capacity remains open despite being posed by van der Meulen in 1971. Each unsolved problem reveals gaps in our understanding of how information flows through networks — making network information theory one of the richest and most challenging areas of the field.

Practice Questions 3 questions

Prerequisite Chain

Understanding ZeroThe Number ZeroCounting to FiveCounting to 10Counting to 20Counting a Set of Objects Up to 20Cardinality: The Last Number CountedMatching Numerals to QuantitiesSubitizing Small QuantitiesAddition Within 10Number Bonds to 10Addition Within 20Doubles and Near DoublesDoubles Facts Within 10Near Doubles Facts Within 20Mental Math Strategies for AdditionMental Math: Adding and Subtracting TensAddition Within 100Repeated Addition as MultiplicationMultiplication as Equal GroupsMultiplication: ArraysBasic Multiplication Facts (0s, 1s, 2s, 5s, 10s)Multiplication Facts Within 100Division as Equal SharingDivision as Grouping (Measurement Division)Division: Grouping (Repeated Subtraction) ModelDivision: Fair Sharing ModelDivision as Equal SharingDivision as GroupingBasic Division FactsDivision Facts Within 100Multiplication and Division Fact FamiliesRelationship Between Multiplication and DivisionDivision Facts as Inverse of MultiplicationRemainders and Quotients in DivisionDivision Word ProblemsMulti-Step Word ProblemsSolving Multi-Step Word ProblemsMultiplication Word ProblemsDivision Word ProblemsIntroduction to Long DivisionFactors and MultiplesPrime and Composite NumbersEquivalent FractionsRelating Fractions and DecimalsDecimal Place ValueIntegers and the Number LineComparing and Ordering IntegersAbsolute ValueAdding IntegersSubtracting IntegersMultiplying IntegersDividing IntegersUnit RatesProportionsPercent ConceptConverting Between Fractions, Decimals, and PercentsOperations with Rational NumbersTwo-Step EquationsSolving Multi-Step EquationsEquations with Variables on Both SidesAngle Pairs: Complementary, Supplementary, and VerticalParallel Lines and TransversalsCorresponding AnglesAlternate Interior AnglesTriangle Angle Sum TheoremExterior Angle TheoremTriangle Inequality TheoremSimilar Triangles: AA SimilaritySimilar Triangles: SSS and SAS SimilarityProportions in Similar TrianglesRight Triangle Trigonometry IntroductionSine, Cosine, and Tangent RatiosTrigonometric Ratios ReviewRadian MeasureConverting Between Degrees and RadiansThe Unit CircleGraphing Sine and CosineGraphing Tangent and Reciprocal Trigonometric FunctionsDerivatives of Trigonometric FunctionsAntiderivativesIndefinite IntegralsBasic Integration RulesRiemann SumsDefinite Integral DefinitionProbability Density Functions and Continuous DistributionsCumulative Distribution FunctionsContinuous Random VariablesProbability Density FunctionsExpected ValueWeak Law of Large NumbersProbability Axioms and RulesConditional ProbabilityLaw of Total ProbabilityBayes' TheoremJoint and Conditional EntropyMutual InformationChannel CapacityBinary Symmetric ChannelChannel Coding TheoremNetwork Information Theory

Longest path: 101 steps · 447 total prerequisite topics

Prerequisites (3)

Leads To (3)