Questions: IPSec: Authentication, Encryption, and VPN Tunneling

Question 1

Two branch offices want to connect securely over the public internet so that all traffic between the office networks is encrypted, including each office's internal IP addresses. Which IPSec mode and component achieves this?

• A. Transport mode with AH — it authenticates the original IP header, hiding the internal addresses
• B. Tunnel mode with ESP — it encrypts the entire original packet and wraps it in a new IP packet with gateway addresses
• C. Transport mode with ESP — it encrypts the payload, leaving only the application data visible to attackers
• D. Tunnel mode with AH — it authenticates the encapsulated packet, ensuring internal addresses cannot be modified

Answer: (B)

Tunnel mode is designed for gateway-to-gateway VPNs. It encrypts the entire original IP packet — including its header containing the internal 10.x.x.x addresses — and wraps it in a new outer IP packet with the gateways' public IP addresses. An eavesdropper on the internet sees only the outer header; the internal addressing is completely hidden. Transport mode (options A and C) only encrypts or authenticates the payload, leaving the original IP header visible — which would expose the internal addresses. AH (option D) provides authentication and integrity but no encryption — confidentiality would not be achieved.

Question 2

IPSec traffic between two hosts fails after one host is placed behind a NAT device. Which explanation is most accurate?

• A. NAT changes the source IP address in the outer IP header, which causes AH integrity verification to fail since AH authenticates the IP header
• B. NAT cannot forward ESP traffic because ESP is not based on TCP or UDP port numbers
• C. IKE phase 1 only works over IPv6, which NAT does not support
• D. Tunnel mode is incompatible with NAT because it adds a second IP header that NAT devices cannot process

Answer: (A)

AH authenticates the entire IP header including the source address. When a NAT device changes the source IP address to route the packet back, the AH integrity check fails because the header no longer matches what was signed. This is why AH and NAT are fundamentally incompatible. ESP has a similar but solvable problem: NAT modifies port numbers (for NAPT), which can interfere with ESP sessions — addressed by NAT-T, which encapsulates ESP inside UDP port 4500. Option B is partially true (ESP is not TCP/UDP-based) but is not the root cause of the authentication failure.

Question 3

IPSec operates at the network layer and can therefore protect any IP-based protocol — UDP, ICMP, routing protocols — not just TCP connections.

• T. True
• F. False

Answer: True

This is the key distinction between IPSec and TLS. TLS secures individual TCP connections at the transport layer. IPSec secures IP packets at the network layer, meaning any protocol that rides on IP — TCP, UDP, ICMP, GRE, OSPF, etc. — is protected. This is why IPSec is used for site-to-site VPNs that must carry all traffic between two networks, not just specific application connections.

Question 4

IPSec tunnel mode is inherently more secure than transport mode because it encrypts more data, including the IP header.

• T. True
• F. False

Answer: False

Security level depends on the cryptographic algorithms and key lengths used, not on whether the mode is tunnel or transport. Both modes can use ESP with the same AES encryption and HMAC authentication. Tunnel mode hides the internal IP headers, which provides *privacy* (an attacker cannot see internal network topology), but this is a confidentiality/metadata property, not a fundamental security strength. Transport mode with strong encryption is more secure than tunnel mode with weak encryption. The choice of mode should be driven by network topology needs, not a false assumption that tunnel mode has stronger cryptography.

Question 5

Why does NAT cause problems for IPSec, and what mechanism does NAT Traversal (NAT-T) use to work around this limitation?

Model answer: NAT devices modify IP addresses (and often port numbers) to route packets between private and public networks. IPSec's AH protocol authenticates the entire IP header including the source address, so any NAT modification breaks the integrity check. ESP avoids modifying authenticated data in the header, but NAPT devices that map ports can still disrupt ESP flows that don't have port numbers. NAT-T solves this by encapsulating the ESP payload inside a UDP packet (on port 4500), giving NAT devices a standard UDP header to modify. The ESP integrity checks are now inside the UDP payload, which NAT does not touch.

NAT-T works by adding a layer of indirection: the ESP packet, which NAT cannot safely modify, is wrapped in a UDP datagram that NAT can handle normally. This is a pragmatic engineering workaround — you sacrifice a small amount of overhead (UDP header per packet) to preserve IPSec's security properties across NAT boundaries. Understanding this interaction helps explain why IKE detects NAT during its negotiation and automatically switches to UDP port 4500 encapsulation when NAT is present.