NETWORKING MASTERY · PHASE 5 · MODULE 20 · WEEK 18
🔒 TLS Internals
TLS 1.3 handshake · Record protocol · Cipher suites · Certificate validation · 0-RTT · mTLS · SSL inspection
Advanced Prerequisite: M19 Cryptography RFC 8446 Ubiquitous — Every HTTPS Connection 2 Labs

TLS — TRANSPORT LAYER SECURITY

🔒

TLS in the Protocol Stack

OVERVIEW

TLS (Transport Layer Security) is the protocol that makes HTTPS, SMTPS, IMAPS, FTPS, and many other "S" protocols secure. It sits between TCP and the application layer, providing: server authentication (via certificates), optional client authentication (mTLS), forward-secret key exchange (ECDHE), and authenticated encryption of all application data.

TLS 1.3 (RFC 8446, 2018) is the current standard. It eliminated all deprecated algorithms, reduced handshake latency from 2 RTTs to 1 RTT (0 RTT for resumption), and simplified the protocol significantly. Understanding TLS 1.3 is essential for NGFW because HTTPS carries the majority of internet traffic — and inspecting it requires understanding or terminating the TLS session.

TLS 1.3 HANDSHAKE — 1 RTT TO ENCRYPTED DATA

🤝

TLS 1.3 Full Handshake

HANDSHAKE
/* TLS 1.3 Handshake — message flow */

CLIENT                                           SERVER
  │                                                │
  │─── ClientHello ────────────────────────────→  │
  │    • client_random (32 bytes)                  │
  │    • supported_versions: [TLS 1.3, TLS 1.2]   │
  │    • supported_groups: [X25519, P-256, P-384]  │
  │    • key_share: X25519 ephemeral pubkey        │
  │    • signature_algs: [Ed25519, ECDSA P-256,    │
  │                        RSA-PSS-SHA256]         │
  │    • server_name (SNI): "example.com"          │
  │    • psk_ke_modes (if resuming)                │
  │                                                │
  │←── ServerHello ────────────────────────────── │
  │    • server_random (32 bytes)                  │
  │    • selected cipher: TLS_AES_256_GCM_SHA384   │
  │    • key_share: X25519 server ephemeral pubkey │
  │    • selected TLS version: 1.3                 │
  │                                                │
  │ [ECDH shared secret computed by both sides]    │
  │ [Handshake keys derived via HKDF]              │
  │ [All subsequent messages are ENCRYPTED]        │
  │                                                │
  │←── {EncryptedExtensions} ─────────────────── │
  │    • ALPN: "h2" (HTTP/2 negotiated)            │
  │    • max_fragment_length, server_cert_type      │
  │                                                │
  │←── {Certificate} ──────────────────────────── │
  │    • Server's X.509 certificate chain          │
  │                                                │
  │←── {CertificateVerify} ────────────────────── │
  │    • Signature over transcript hash            │
  │      (proves server has private key)           │
  │                                                │
  │←── {Finished} ─────────────────────────────── │
  │    • HMAC over entire handshake transcript     │
  │      (proves handshake integrity)              │
  │                                                │
  │─── {Finished} ──────────────────────────────→ │
  │    • Client's HMAC over transcript             │
  │                                                │
  │←→  {Application Data (AEAD encrypted)} ←────→ │

RTT count: 1 full RTT before application data can flow
           (ClientHello → ServerHello+Cert+Finished → ClientFinished+AppData)

💡 Key insight: In TLS 1.3, the server can send encrypted extensions, its certificate, and its Finished message all in one flight — before receiving anything from the client beyond ClientHello. This is possible because ECDHE allows the server to derive encryption keys immediately after seeing the client's key share. The client verifies the server's Finished HMAC to confirm the handshake wasn't tampered with.

TLS 1.3 KEY SCHEDULE — HOW KEYS ARE DERIVED

🗝️

HKDF-Based Key Schedule

KEY SCHEDULE
/* TLS 1.3 Key Schedule (RFC 8446 §7.1) */
/* All derivations use HKDF with the negotiated hash (SHA-256 or SHA-384) */

0 (Early Secret)
  ├─ Early Traffic Keys (for 0-RTT data, if resuming with PSK)
  │
  ↓ HKDF-Extract(PSK or 0-bytes, Early Secret)
Handshake Secret
  ├─ client_handshake_traffic_secret
  │   → client_handshake_key (AES key for client→server during handshake)
  │   → client_handshake_iv  (nonce base)
  ├─ server_handshake_traffic_secret
  │   → server_handshake_key (AES key for server→client during handshake)
  │   → server_handshake_iv
  │
  ↓ HKDF-Extract(ECDHE shared secret, Handshake Secret)
Master Secret
  ├─ client_application_traffic_secret_0
  │   → client_write_key (AES key for client→server application data)
  │   → client_write_iv
  ├─ server_application_traffic_secret_0
  │   → server_write_key (AES key for server→client application data)
  │   → server_write_iv
  ├─ exporter_master_secret (for channel binding)
  └─ resumption_master_secret (for session tickets / PSK resumption)

/* Nonce construction — prevents nonce reuse */
/* For each record: nonce = write_iv XOR sequence_number (64-bit, left-padded) */
/* Sequence number increments with each record → unique nonce per record */

/* Key update (post-handshake) */
/* Either side can send KeyUpdate message → derive new traffic keys */
new_secret = HKDF-Expand-Label(current_secret, "traffic upd", "", hash_len)
/* Forward secrecy within a session: old keys deleted, new keys derived */

TLS RECORD PROTOCOL — WIRE FORMAT

📦

TLS Record Structure

RECORD FORMAT
/* TLS Record format (all TLS versions) */
+------------------+------------------+------------------+
| Content Type (1B)| Version (2B)     | Length (2B)      |
+------------------+------------------+------------------+
| Payload (up to 16384 bytes)                            |
+--------------------------------------------------------+

Content Types:
  20 = change_cipher_spec (legacy, sent for TLS 1.2 compat)
  21 = alert             (error notification)
  22 = handshake         (ClientHello, ServerHello, Certificate, etc.)
  23 = application_data  (encrypted payload)

Version field in TLS 1.3:
  Outer record: 0x0303 (TLS 1.2) — for middlebox compatibility
  Inner content_type (inside AEAD ciphertext): real type

/* TLS 1.3 Application Data record layout */
+------+--------+--------+----------------------------------+----------+
| 0x17 | 0x0303 | length | Encrypted(application_data +     | auth_tag |
|  23  | TLS1.2 | 2B     | inner_content_type) — AEAD       | 16B      |
+------+--------+--------+----------------------------------+----------+

/* AEAD inputs for encrypting a record */
Plaintext:  application_data bytes + inner_content_type (1 byte at end)
AAD:        TLS record header (5 bytes: type + version + length)
Key:        write_key (from key schedule)
Nonce:      write_iv XOR (seq_number as 12-byte big-endian)

/* Maximum record size */
16384 bytes (2^14) of plaintext per record
+ 256 bytes of padding (optional, hides true record size)
+ 16 bytes auth tag
= up to 16657 bytes per record

/* Alert record format (2 bytes inside TLS record) */
Level:       1=warning, 2=fatal
Description: 0=close_notify, 10=unexpected_message, 20=bad_record_mac,
             42=bad_certificate, 48=unknown_ca, 70=protocol_version,
             80=internal_error, 100=no_renegotiation, 112=unrecognized_name (SNI)

/* Wireshark TLS decryption */
SSLKEYLOGFILE=/tmp/keys.log curl https://example.com
# In Wireshark: Edit → Preferences → TLS → Master-Secret log file → /tmp/keys.log
# Wireshark will decrypt all TLS records and show plaintext handshake + data

TLS 1.2 vs TLS 1.3 — KEY DIFFERENCES

📊

What Changed from TLS 1.2 to 1.3

COMPARISON
FeatureTLS 1.2TLS 1.3
Handshake RTTs2 RTTs minimum1 RTT (0 RTT for resumption)
Forward secrecyOptional (ECDHE or static RSA)Mandatory (ECDHE always)
Cipher suitesHundreds incl. RC4, 3DES, NULL, anon5 only, all AEAD
Key exchangeRSA, ECDHE, DHE, ECDH (static)ECDHE, DHE (finite field) only
Certificate encryptionPlaintext in handshake (visible to network)Encrypted (after server key derived)
RenegotiationAllowed (caused vulnerabilities)Removed entirely
CompressionOptional (CRIME attack)Removed
MAC-then-EncryptUsed in CBC mode (BEAST, POODLE)Removed — AEAD only
Session IDsServer stores session stateStateless session tickets only
SNI encryptionNo (SNI in cleartext ClientHello)ECH (Encrypted ClientHello) — draft
Removed from TLS 1.3RSA key exchange, CBC, RC4, 3DES, MD5, SHA-1, renegotiation, compression, DSA

TLS 1.3 Cipher Suites — Only 5

TLS_AES_128_GCM_SHA256          (most common, high performance)
TLS_AES_256_GCM_SHA384          (higher security)
TLS_CHACHA20_POLY1305_SHA256    (mobile/ARM performance)
TLS_AES_128_CCM_SHA256          (constrained IoT)
TLS_AES_128_CCM_8_SHA256        (constrained IoT, shorter tag)

# Note: no key exchange or auth in TLS 1.3 cipher suites
# Key exchange is always ECDHE (negotiated separately in supported_groups)
# Authentication is always certificate-based (negotiated in signature_algs)

0-RTT AND SESSION RESUMPTION

PSK and 0-RTT Early Data

0-RTT
/* TLS 1.3 session resumption via PSK (Pre-Shared Key) */

After a successful TLS 1.3 handshake, the server sends a NewSessionTicket:
  - Contains a PSK (pre-shared key) encrypted with a server-only ticket key
  - Includes a ticket_lifetime (e.g., 7 days)
  - Client stores this opaque blob

On reconnect, client includes the PSK in ClientHello:
  - pre_shared_key extension: ticket blob
  - psk_key_exchange_modes: psk_dhe_ke (PSK + ephemeral DH — recommended)
                            or psk_ke (PSK only — no forward secrecy!)
  - early_data extension: client wants to send 0-RTT data

/* 0-RTT early data — zero round-trip cost */

Standard 1-RTT:     ClientHello → ServerHello+Cert+Finished → {AppData}
0-RTT resumption:   ClientHello + {EarlyData} → ServerHello → {AppData}
                    ↑ Application data piggybacks on ClientHello!

/* 0-RTT security limitations */
Replay attack risk:
  Attacker captures ClientHello+EarlyData, replays it to server.
  Server has no way to distinguish replay from original!
  
  Mitigations:
  1. Only use 0-RTT for idempotent requests (GET, not POST)
  2. Server-side replay detection (store nonces, use anti-replay window)
  3. Accept risk for non-sensitive use (performance vs security tradeoff)

0-RTT does NOT provide forward secrecy for early data:
  If PSK ticket key is compromised → early data decryptable
  Post-handshake application data DOES have forward secrecy (ECDHE)

/* NGFW considerations for 0-RTT */
# 0-RTT early data is encrypted with the early_traffic_key
# Without the PSK or TLS session keys, NGFW cannot decrypt 0-RTT
# SSL inspection proxy must handle 0-RTT specially
# Many NGFW products simply reject 0-RTT by not returning early_data in EncryptedExtensions

mTLS — MUTUAL AUTHENTICATION

🤝

Client Certificate Authentication

mTLS
/* Standard TLS: only server is authenticated */
/* mTLS (mutual TLS): both server AND client present certificates */

/* mTLS handshake additions */
After sending Certificate + CertificateVerify + Finished, server sends:
  CertificateRequest: list of acceptable CA DNs for client certificates

Client responds with:
  Certificate: client's X.509 certificate (or empty if none available)
  CertificateVerify: signature over handshake transcript with client private key
  Finished: as normal

/* Use cases for mTLS */
Service mesh (Istio, Linkerd): all microservices authenticate each other
Zero-trust networks: every connection requires client cert (device identity)
API security: client apps authenticate with cert instead of API keys
IoT devices: device certificates for mutual auth to backend
NGFW policy: require client cert for access to sensitive internal resources

/* Configure nginx for mTLS */
ssl_client_certificate /etc/ssl/ca.pem;  # CA that signed client certs
ssl_verify_client on;                     # require client cert
ssl_verify_depth 2;                       # allow one intermediate CA

/* OpenSSL mTLS server in C */
SSL_CTX *ctx = SSL_CTX_new(TLS_server_method());
SSL_CTX_load_verify_locations(ctx, "ca.pem", NULL);
SSL_CTX_use_certificate_file(ctx, "server.pem", SSL_FILETYPE_PEM);
SSL_CTX_use_PrivateKey_file(ctx, "server.key", SSL_FILETYPE_PEM);
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT, NULL);

/* After SSL_accept(): inspect client certificate */
X509 *client_cert = SSL_get_peer_certificate(ssl);
X509_NAME *subj = X509_get_subject_name(client_cert);
char cn[256];
X509_NAME_get_text_by_NID(subj, NID_commonName, cn, sizeof(cn));
printf("Client cert CN: %s\n", cn);
X509_free(client_cert);

SSL INSPECTION — NGFW TLS INTERCEPTION

🔬

How SSL/TLS Inspection Works

SSL INSPECTION
/* SSL inspection (TLS MITM proxy) — the NGFW's view */

Normal TLS:
  Client ←── TLS ──→ Server
  Client trusts server's cert from a real CA
  NGFW sees: encrypted bytes → cannot inspect content

SSL inspection:
  Client ←── TLS ──→ NGFW ←── TLS ──→ Server
  
  NGFW-Server leg:
    NGFW establishes TLS to the real server
    Validates server's real certificate
    NGFW has the session keys → can decrypt/inspect server responses
  
  Client-NGFW leg:
    NGFW generates a certificate for the domain
    Signs it with the corporate CA (deployed to all managed devices)
    Client validates against corporate CA → succeeds
    NGFW has these session keys too → can decrypt/inspect client requests

/* What SSL inspection reveals */
Full HTTP URL path (not just hostname)
All HTTP request/response headers
Request bodies (POST data, form submissions)
Response bodies (file downloads → malware scanning)
WebSocket data
gRPC payloads

/* What SSL inspection breaks */
Certificate pinning: apps that pin to specific certs (Twitter app, many banking apps)
HPKP (deprecated): HTTP Public Key Pinning
Client certificates (mTLS): NGFW must handle client cert forwarding
QUIC/HTTP3: QUIC encrypts more aggressively, harder to intercept

/* NGFW SSL inspection bypass list (do NOT inspect) */
Banking domains (privacy regulation)
Healthcare portals (HIPAA)
Legal/HR applications (attorney-client privilege)
Apps known to use certificate pinning
Internal PKI-protected services (use different trust chain)

/* Implementing basic TLS termination in C with OpenSSL */
SSL_CTX *server_ctx = SSL_CTX_new(TLS_server_method());
/* Load your generated cert for the target domain */
SSL_CTX_use_certificate(server_ctx, generated_cert);
SSL_CTX_use_PrivateKey(server_ctx, generated_key);

SSL_CTX *client_ctx = SSL_CTX_new(TLS_client_method());
/* Connect to real server */
SSL *client_ssl = SSL_new(client_ctx);
SSL_set_fd(client_ssl, server_socket_fd);
SSL_connect(client_ssl);
/* Verify real server cert */
X509 *real_cert = SSL_get_peer_certificate(client_ssl);
/* Extract domain, generate matching cert for client, serve it */

⚠️ ECH (Encrypted ClientHello) — in development for TLS 1.3 — will encrypt the SNI extension and other ClientHello fields, preventing NGFW from seeing the destination hostname without decrypting the entire TLS session. This fundamentally challenges SNI-based filtering and makes SSL inspection the only way to identify destinations. Watch RFC drafts for ECH deployment timeline.

LAB 1

TLS 1.3 Handshake Dissection

Objective: Capture and fully decode a TLS 1.3 handshake using Wireshark with TLS key logging.

1
Set up key logging: export SSLKEYLOGFILE=/tmp/tls_keys.log. Make a TLS 1.3 connection: curl --tlsv1.3 -v https://cloudflare.com 2>&1 | head -40. Capture simultaneously: sudo tcpdump -i eth0 -w /tmp/tls.pcap host cloudflare.com. Open pcap in Wireshark with key log configured.
2
In Wireshark, examine the ClientHello: find and record — the supported_versions extension (should include 0x0304 = TLS 1.3), the key_share extension (X25519 public key bytes), the server_name extension (SNI), the signature_algorithms extension. Note that all these are in the clear.
3
Examine the ServerHello: verify TLS 1.3 is selected (version extension 0x0304), find the server's X25519 key share. After the ServerHello, all subsequent messages should show as "Encrypted Handshake Message" without the key log, but decryptable with the key log. Verify the Certificate record shows the real cert chain.
4
Verify the cipher suite negotiated (should be TLS_AES_256_GCM_SHA384 or TLS_AES_128_GCM_SHA256). Find the application data records — without key log they're opaque; with key log Wireshark shows the HTTP/2 frames inside. Count the total number of TLS records in the handshake and confirm the 1-RTT timing.
LAB 2

Build a TLS Client and Server with OpenSSL

Objective: Implement a TLS 1.3 echo server and client in C using OpenSSL. Extend with mTLS client authentication.

1
Generate test certificates: openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -keyout server.key -out server.crt -days 365 -nodes -subj "/CN=localhost". Write a minimal TLS server: socket → SSL_CTX_new(TLS_server_method()) → SSL_CTX_use_certificate/PrivateKey → accept loop → SSL_accept → SSL_read/write.
2
Force TLS 1.3 only: SSL_CTX_set_min_proto_version(ctx, TLS1_3_VERSION). Verify with: openssl s_client -connect localhost:8443 -tls1_3. Confirm the cipher suite selected. Check the certificate presented: openssl s_client -connect localhost:8443 2>/dev/null | openssl x509 -noout -text.
3
Add mTLS: generate a client CA and client certificate. Configure the server to request a client cert (SSL_VERIFY_PEER). Test with: openssl s_client -connect localhost:8443 -cert client.crt -key client.key. Verify the server prints the client cert's CN. Try without a cert — verify the server rejects with "alert handshake failure".
4
Enable session tickets and test resumption: after a successful connection, store the session: SSL_get1_session(ssl). On the next connection, restore it: SSL_set_session(ssl, session). Capture both connections in Wireshark and verify the second one is shorter (no Certificate/CertificateVerify).

M20 MASTERY CHECKLIST

When complete: Move to M21 - IPsec and IKEv2 — the VPN protocol stack used for site-to-site and remote access in enterprise networks, and the encryption layer for many NGFW deployments.

← M19 Cryptography 🗺️ Roadmap Next: M21 - IPsec →