VPN Protocols: WireGuard, IPsec, OpenVPN & Cryptographic Tunneling

Deep dive into virtual private networks, cryptographic encapsulation, Diffie-Hellman key exchange, IPsec AH/ESP, WireGuard Noise protocol, and split tunneling.

Cryptographic Encapsulation & Tunnel Mechanics

A Virtual Private Network (VPN) creates an encrypted tunnel across untrusted public networks, encapsulating private IP packets inside routable public IP packets with authenticated encryption.

WireGuard Protocol:State-of-the-art VPN protocol built with only ~4,000 lines of code, utilizing Noise IK handshake, Curve25519, ChaCha20-Poly1305, and BLAKE2s.
IPsec (ESP & AH):Encapsulating Security Payload (ESP) provides confidentiality and authentication; Authentication Header (AH) provides integrity without encryption.
OpenVPN (SSL/TLS):Runs in user space using OpenSSL for encryption, operating over UDP (port 1194) or TCP (port 443) for firewall traversal.

Ephemeral Key Exchange: Peers perform a Diffie-Hellman handshake (Curve25519) to derive symmetric session keys with Perfect Forward Secrecy (PFS).
Packet Encapsulation: The original IP packet is encrypted with ChaCha20 and authenticated with Poly1305 MAC tag, then wrapped in an outer UDP header.
Ingress Decapsulation: The receiving endpoint verifies the Poly1305 tag, decrypts the payload, and forwards the inner packet onto the private subnet interface.

Definition: Perfect Forward Secrecy (PFS)

A cryptographic property ensuring that the compromise of long-term private keys does not compromise past session keys derived from ephemeral Diffie-Hellman exchanges.

Diffie-Hellman Shared Secret Derivation

K = (g^a mod p)^b mod p = g^(ab) mod p

Variable Definitions & Term Breakdown:

g, pPublic base generator and prime modulus

a, bAlice and Bob's private ephemeral secrets

KIdentical shared secret key derived without transmitting a or b

VPN Protocols Comparison Matrix

Protocol	Codebase Size	Primary Ciphers	Handshake Latency	Throughput
WireGuard	~4,000 LOC	ChaCha20-Poly1305, Curve25519	1 RTT (~10ms)Optimal	Ultra High (Line Rate)
IPsec / IKEv2	~100,000 LOC	AES-GCM, SHA-2, DH groups	2-4 RTTs	High
OpenVPN	~120,000 LOC	OpenSSL library	3-5 RTTs	Moderate

Performance, cryptographic modernness, and codebase footprint

Tip

For modern infrastructure, prefer WireGuard over OpenVPN. WireGuard runs in the Linux kernel, avoiding user-space context switches and delivering up to 4x higher throughput.

PythonDiffie-Hellman Key Agreement Implementation

1# Simplified Diffie-Hellman Key Exchange Simulation
2p = 23  # Shared public prime
3g = 5   # Shared public base
4 
5# Alice's private key
6a_private = 6
7A_public = (g ** a_private) % p
8 
9# Bob's private key
10b_private = 15
11B_public = (g ** b_private) % p
12 
13# Shared secret derivation
14Alice_shared = (B_public ** a_private) % p
15Bob_shared = (A_public ** b_private) % p
16 
17print(f'Alice Public: {A_public}, Bob Public: {B_public}')
18print(f'Alice Secret Key: {Alice_shared}')
19print(f'Bob Secret Key:   {Bob_shared}')
20print(f'Keys Match? {Alice_shared == Bob_shared}')

Output

Alice Public: 8, Bob Public: 19
Alice Secret Key: 2
Bob Secret Key:   2
Keys Match? True

INIWireGuard Interface Configuration (wg0.conf)

1[Interface]
2Address = 10.0.0.1/24
3PrivateKey = aGVs... (256-bit base64)
4ListenPort = 51820
5 
6[Peer]
7PublicKey = d29y... (Peer's base64 pubkey)
8AllowedIPs = 10.0.0.2/32
9Endpoint = 203.0.113.50:51820

Output

[WireGuard Tunnel Active: 10.0.0.1 <-> 10.0.0.2]

Quick Syntax & Reference Sheet

Concept / Rule	Syntax / Usage
WireGuard Up	`wg-quick up wg0`
Keygen	`wg genkey \| tee privatekey \| wg pubkey > publickey`

Common Pitfalls & Developer Mistakes

✕ Mistake 1:

Using TCP over TCP (TCP meltdown) with OpenVPN

Why it happens: When running TCP-based VPN tunnels over a TCP transport, packet loss triggers nested exponential retransmissions, collapsing throughput.

Correction: Always use UDP as the underlying transport protocol for VPN tunnels.

# In OpenVPN config:
proto udp
port 1194

Frequently Asked Questions

What is Split Tunneling?

Split Tunneling allows certain traffic (e.g. corporate subnets) to route through the encrypted VPN tunnel while general Internet browsing exits directly via the local gateway.