A BSC has crossover probability p = 0.5. What is the channel capacity, and what does this mean physically?
AC = 0.5 bits — you can transmit at half the rate of a noiseless channel
BC = 0 bits — the output is completely independent of the input, so no information passes through
CC = 1 bit — noise has no effect because the channel is symmetric
DC = -1 bits — the channel inverts all bits
When p = 0.5, each output bit is equally likely to be 0 or 1 regardless of the input — the channel is pure noise. H(0.5) = 1, so C = 1 - 1 = 0 bits. No communication is possible. The output is statistically independent of the input: knowing the output tells you nothing about what was sent. Interestingly, p = 1 (all bits inverted) gives C = 1 - H(1) = 1 - 0 = 1 bit — a perfect inverter is as good as a perfect channel because the receiver simply flips every bit.
Question 2 Multiple Choice
A BSC has crossover probability p = 0.1. The capacity is C = 1 - H(0.1) ≈ 0.531 bits per use. A naive scheme sends each message bit once with no coding. What is its effective reliable rate?
A0.531 bits per use — the same as capacity
B0.9 bits per use — since 90% of bits arrive correctly
C0 bits per use reliably — uncoded transmission has a 10% bit error rate, which means information is unreliable
D1 bit per use — each channel use carries one bit regardless of errors
Without coding, each bit has a 10% chance of error. While 90% arrive correctly, there is no way for the receiver to know WHICH bits are wrong. 'Reliable' communication means error probability approaching zero, not just low. Uncoded transmission at any positive rate has non-vanishing error probability on a noisy channel. The channel coding theorem says you CAN achieve rates up to 0.531 bits/use with vanishing error — but only with error-correcting codes that add redundancy.
Question 3 Short Answer
The binary entropy function H(p) is symmetric around p = 0.5 and reaches its maximum of 1 bit at p = 0.5. Explain why BSC capacity C = 1 - H(p) is also symmetric around p = 0.5 and what this means for p > 0.5.
Think about your answer, then reveal below.
Model answer: H(p) = H(1-p), so C(p) = 1 - H(p) = 1 - H(1-p) = C(1-p). The capacity is symmetric around p = 0.5. For p > 0.5, the channel flips bits more often than not — it is a 'mostly-inverting' channel. But an inverter is still informative: if p = 0.9, the receiver simply flips every received bit mentally, obtaining the equivalent of a channel with p = 0.1. The receiver can exploit systematic inversion just as easily as systematic preservation. Only at p = 0.5 — where flipping is equally likely as non-flipping — is the channel truly useless.
This symmetry is specific to the BSC. It shows that what matters for capacity is not the raw error rate but the predictability of the channel's behavior. Both very low and very high error rates are highly predictable (bits usually preserved vs. bits usually flipped), leaving capacity high. Only maximum randomness (p = 0.5) destroys all information.