The Hidden Cost of "Hello, Server"
MENTAL MODELBefore your code sends a single byte of application data, the OS and network stack silently complete three distinct protocol exchanges. A backend engineer who doesn't understand these layers will misdiagnose latency, misconfig TLS, and write servers that break under load.
This module gives you full mental ownership of the connection lifecycle — from the moment a hostname is typed to the moment encrypted application data flows.
Connection Lifecycle: Cold Start
Client DNS Resolver TCP Stack TLS Stack Server App
│ │ │ │ │
│──── getaddrinfo("api.io") ──▶│ │ │ │
│ (recursive query chain) │ │ │ │
│◀─── IP: 1.2.3.4 ────────────│ │ │ │
│ │ │ │ │
│──── SYN ──────────────────────────────────────────────────────────────────────────────▶
│◀─── SYN-ACK ──────────────────────────────────────────────────────────────────────────
│──── ACK ──────────────────────────────────────────────────────────────────────────────▶
│ │ │ TCP ESTABLISHED│ │
│──── ClientHello ──────────────────────────────────────────────────────────────────────▶
│◀─── ServerHello + Cert + Finished ────────────────────────────────────────────────────
│──── Finished ─────────────────────────────────────────────────────────────────────────▶
│ │ │ │ TLS ESTABLISHED │
│══════════════════════ Encrypted Application Data ═══════════════════════════════════▶│
Latency Budget: Cold vs Warm
| Phase | Cold start | Warm (cached/reused) | Where saved |
|---|---|---|---|
| DNS resolution | 20–100 ms | 0 ms | OS/stub resolver TTL cache |
| TCP handshake | 1 × RTT | 0 (connection pool) | Connection pooling / keep-alive |
| TLS 1.3 handshake | 1 × RTT | 0 (0-RTT resumption) | Session tickets / PSK |
| First byte of response | 1 × RTT | 1 × RTT | Always paid |
| Total cold | ≈ 3–4 RTT + DNS. On 50 ms RTT link: ~200 ms before any data. | ||
Key insight: TLS 1.3 reduced handshake cost from 2 RTT (TLS 1.2) to 1 RTT — and 0-RTT resumption eliminates it entirely for repeat connections. This is why upgrading TLS version has measurable user-facing impact.
Why Each Layer Exists
- 1DNS — humans use names; routers use IPs. DNS is the distributed phonebook that maps one to the other. It also carries routing policy (round-robin, geo, health-check failover) via multiple A/AAAA records.
- 2TCP — IP is unreliable and unordered. TCP adds reliability (retransmission), ordering (sequence numbers), and flow + congestion control. The 3-way handshake establishes shared state (ISNs) before data flows.
- 3TLS — TCP provides delivery but not privacy or authenticity. TLS negotiates cipher suites, authenticates the server via certificates, and derives symmetric session keys — turning a transparent pipe into an encrypted tunnel.
DNS Architecture
DISTRIBUTED SYSTEMDNS is a globally distributed, hierarchical, eventually-consistent key-value store. It is the largest distributed database on the internet. Understanding its resolution chain is essential for diagnosing outages and designing resilient services.
Recursive Resolution Chain
Browser/App Stub Resolver Recursive Resolver Root NS TLD NS (.io) Auth NS (api.io)
│ │ │ │ │ │
│─ gethostbyname() ──▶│ │ │ │ │
│ │──── Query: api.io ─▶│ │ │ │
│ │ │── Who knows .io? ─▶ │ │
│ │ │◀─ Try ns1.nic.io ── │ │
│ │ │──────────────────── api.io A? ───▶│ │
│ │ │◀─────────────────── Try ns1.api ──│ │
│ │ │──────────────────────────────────── api.io A? ─────▶
│ │ │◀─────────────────────────────────── 1.2.3.4 TTL=300
│◀──── 1.2.3.4 ───────│◀──── 1.2.3.4 ──────│ │ │ │
│ │ (cached for 300s) │ │ │ │
🏢 Analogy: Ask reception (stub resolver) for "Bob in Engineering". Reception calls the central operator (recursive resolver) who consults the building directory (root), which points to the floor directory (.io TLD), which finally has Bob's desk number (authoritative NS returns the IP).
DNS Record Types
| Type | Purpose | Example value | Backend use |
|---|---|---|---|
| A | IPv4 address | 1.2.3.4 | Server IP resolution |
| AAAA | IPv6 address | 2001:db8::1 | Dual-stack support |
| CNAME | Canonical alias | api.io → lb-123.aws.com | CDN, load balancer aliasing |
| MX | Mail exchange | 10 mail.api.io | Email routing |
| TXT | Arbitrary text | v=spf1 include:... | SPF/DKIM, ownership proof |
| SRV | Service location | _http._tcp 80 host | Service discovery (gRPC/K8s) |
| PTR | Reverse lookup | 4.3.2.1.in-addr.arpa | Log enrichment, spam checks |
| NS | Authoritative nameservers | ns1.cloudflare.com | Delegation chain |
| SOA | Zone authority | Serial, refresh, retry, expire | Zone transfer, negative TTL |
DNS Message Format
Wire Format (RFC 1035)
BINARY PROTOCOLDNS messages are binary, typically UDP (port 53), falling back to TCP for responses > 512 bytes or zone transfers. Each message has a fixed 12-byte header followed by variable sections.
| Section | Size | Content |
|---|---|---|
| Header | 12 bytes | ID (2B), Flags (2B), QDCOUNT, ANCOUNT, NSCOUNT, ARCOUNT (2B each) |
| Question | variable | QNAME (label encoding), QTYPE (2B), QCLASS (2B) |
| Answer | variable | NAME, TYPE, CLASS, TTL (4B), RDLENGTH, RDATA |
| Authority | variable | NS records for zone delegation |
| Additional | variable | Glue records (A for the NS itself) |
Label encoding: api.io becomes \x03api\x02io\x00 — each label prefixed with its length byte, terminated with zero byte. DNS uses pointer compression (2-byte offset) to avoid repeating names.
TTL and Caching Behaviour
- 1Stub resolver (in libc) — caches based on TTL.
nscdorsystemd-resolvedmay add another caching layer. Callgetaddrinfo()— never roll your own DNS in production. - 2Negative caching (NXDOMAIN) — the SOA record's MINIMUM field caps negative TTL. A wrong hostname lookup causes a 60-second penalty per resolver, per negative TTL.
- 3TTL strategy: During normal operation use 300s–3600s. During deployments or planned failovers, lower TTL to 30–60s before the change, then restore after.
Security: DNS Cache Poisoning
Kaminsky Attack (2008)
ATTACK VECTORAn attacker can inject forged DNS responses by racing the legitimate response. Classic UDP DNS used predictable transaction IDs (16-bit, ~65K space) — an attacker sending spoofed responses for all 65K IDs had a good chance of winning the race.
Defences
- Source port randomisation — expands guessing space from 65K to 65K × 65K ≈ 4 billion
- DNSSEC — cryptographic signatures on DNS records; validates chain of trust from root to zone
- DNS-over-HTTPS (DoH) / DNS-over-TLS (DoT) — prevents on-path eavesdropping and tampering
- 0x20 encoding — randomise case in query name; attacker must match exact case in response
C Code: getaddrinfo()
#include <sys/types.h> #include <sys/socket.h> #include <netdb.h> #include <arpa/inet.h> #include <stdio.h> #include <string.h> /* Resolve hostname → IP(s), prefer IPv4 */ int resolve_host(const char *host, char out_ip[INET6_ADDRSTRLEN]) { struct addrinfo hints, *res, *rp; memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_UNSPEC; /* IPv4 or IPv6 */ hints.ai_socktype = SOCK_STREAM; /* TCP */ int rc = getaddrinfo(host, NULL, &hints, &res); if (rc != 0) { fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rc)); return -1; } for (rp = res; rp != NULL; rp = rp->ai_next) { void *addr; if (rp->ai_family == AF_INET) { struct sockaddr_in *ipv4 = (struct sockaddr_in *)rp->ai_addr; addr = &ipv4->sin_addr; } else { struct sockaddr_in6 *ipv6 = (struct sockaddr_in6 *)rp->ai_addr; addr = &ipv6->sin6_addr; } inet_ntop(rp->ai_family, addr, out_ip, INET6_ADDRSTRLEN); printf("Resolved %s → %s\n", host, out_ip); break; /* take first result */ } freeaddrinfo(res); return 0; }
⚠️ Never call gethostbyname() — it is not thread-safe (returns pointer to static buffer), doesn't support IPv6, and is deprecated in POSIX.1-2008. Always use getaddrinfo().
Why a 3-Way Handshake?
RELIABILITYTCP is connection-oriented: both sides must agree on initial sequence numbers (ISNs) before data can flow. The 3-way handshake achieves this with the minimum number of round trips needed to confirm bidirectional reachability and synchronise state.
A 2-way handshake would suffice for the client to know the server is reachable — but the server wouldn't know the client received the SYN-ACK. The third ACK closes this gap.
3-Way Handshake in Detail
Client (CLOSED) Server (LISTEN)
│ │
│── SYN (seq=ISN_c, SYN=1) ──────────────────────────▶
│ [client → SYN_SENT] │ [server → SYN_RCVD]
│ │
│◀── SYN-ACK (seq=ISN_s, ack=ISN_c+1, SYN=1,ACK=1)──│
│ [client → ESTABLISHED] │
│ │
│── ACK (seq=ISN_c+1, ack=ISN_s+1, ACK=1) ───────────▶
│ [server → ESTABLISHED]
│ │
│══════════════ DATA FLOWS ══════════════════════════▶│
Initial Sequence Number (ISN) Randomness
Why ISNs Must Be Random
SECURITYEarly TCP implementations used predictable ISNs (incrementing counters). This allowed TCP session hijacking: an attacker who could predict the server's ISN could forge an ACK and inject data into a connection without being on the network path.
Modern kernels (Linux, BSD) use ISNs derived from a keyed hash of the 4-tuple (src-ip, src-port, dst-ip, dst-port) plus a secret key and timestamp — making ISNs unpredictable while still monotonically increasing within a connection.
TCP Options Negotiated During Handshake
| Option | Kind | Purpose | Default if absent |
|---|---|---|---|
| MSS | 2 | Maximum Segment Size — largest payload per segment, usually 1460 on Ethernet (1500 MTU − 40B IP+TCP) | 536 bytes (safe minimum) |
| SACK | 4/5 | Selective Acknowledgement — receiver tells sender exactly which segments arrived; avoids retransmitting already-received data | Go-Back-N (retransmit from gap) |
| Window Scale | 3 | Shifts the 16-bit window field left by N bits, allowing windows up to 1 GB. Essential for high-bandwidth long-distance links (bandwidth-delay product) | 64 KB max window |
| Timestamps | 8 | RTT measurement + PAWS (Protection Against Wrapped Sequence numbers). Also mitigates blind RST injection | No RTT measurement from headers |
Connection Teardown: FIN vs RST
Graceful: FIN (4-way)
Active closer Passive closer
│ │
│── FIN ────────────────▶ [CLOSE_WAIT]
│ [FIN_WAIT_1] │
│◀── ACK ────────────────│
│ [FIN_WAIT_2] │ (may send more data)
│◀── FIN ────────────────│ [LAST_ACK]
│ [TIME_WAIT] │
│── ACK ────────────────▶│ [CLOSED]
│ (wait 2×MSL) │
│ [CLOSED] │Half-close allows server to finish sending before closing.
Abortive: RST
RST immediately terminates connection — no graceful drain. Caused by:
- Port not listening (
Connection refused) SO_LINGERwithl_linger=0- Out-of-window segment received
- Application crash without
close() - Firewall/middlebox injecting RST
RST causes ECONNRESET on the peer's next read/write.
Key Socket Options
| Option | Level | Effect | When to use |
|---|---|---|---|
SO_REUSEADDR | SOL_SOCKET | Allows bind to a port in TIME_WAIT state. Essential for servers that restart quickly | Always set on server sockets |
SO_REUSEPORT | SOL_SOCKET | Multiple sockets can bind same port; kernel load-balances incoming connections across them | Multi-process/multi-thread servers |
TCP_NODELAY | IPPROTO_TCP | Disables Nagle's algorithm — sends small packets immediately rather than buffering | Interactive protocols (SSH, Redis), latency-sensitive RPCs |
SO_KEEPALIVE | SOL_SOCKET | OS sends keepalive probes after idle period to detect dead peers | Long-lived connections (DB pools) |
TCP_FASTOPEN | IPPROTO_TCP | Send data in SYN packet on repeat connections — saves 1 RTT | Latency-critical repeat connections |
SO_LINGER | SOL_SOCKET | Controls close() behaviour: wait for drain vs send RST immediately | Set l_linger=0 only when intentionally aborting |
C: TCP Server Skeleton
#include <sys/socket.h> #include <netinet/in.h> #include <netinet/tcp.h> #include <unistd.h> #include <stdio.h> #include <string.h> int main(void) { /* 1. Create socket */ int server_fd = socket(AF_INET, SOCK_STREAM, 0); /* 2. Allow port reuse (survive TIME_WAIT on restart) */ int opt = 1; setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)); setsockopt(server_fd, SOL_SOCKET, SO_REUSEPORT, &opt, sizeof(opt)); /* 3. Bind to port 8080, all interfaces */ struct sockaddr_in addr = { .sin_family = AF_INET, .sin_port = htons(8080), .sin_addr.s_addr = INADDR_ANY }; bind(server_fd, (struct sockaddr *)&addr, sizeof(addr)); /* 4. Mark as passive; backlog=128 = max pending SYNs in accept queue */ listen(server_fd, 128); printf("Listening on :8080\n"); while (1) { /* 5. Accept — blocks until 3-way handshake completes */ struct sockaddr_in client_addr; socklen_t client_len = sizeof(client_addr); int conn_fd = accept(server_fd, (struct sockaddr *)&client_addr, &client_len); /* 6. Disable Nagle for low-latency responses */ setsockopt(conn_fd, IPPROTO_TCP, TCP_NODELAY, &opt, sizeof(opt)); /* 7. Read request (simplified — real code loops until \r\n\r\n) */ char buf[4096]; ssize_t n = recv(conn_fd, buf, sizeof(buf) - 1, 0); buf[n] = '\0'; /* 8. Send response */ const char *resp = "HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nHello"; send(conn_fd, resp, strlen(resp), 0); /* 9. Graceful close — sends FIN, drains */ close(conn_fd); } }
accept() backlog: The backlog parameter to listen() limits the number of completed-but-not-yet-accepted connections in the kernel's accept queue. Under SYN flood, the incomplete SYN queue fills first. Set net.ipv4.tcp_syncookies=1 to handle SYN floods without dropping legitimate connections.
TCP State Machine
11 STATESTCP is a finite state machine. Each connection independently transitions through states based on segments received and API calls made. Knowing these states helps you diagnose stuck connections, TIME_WAIT accumulation, and FIN_WAIT_2 leaks using ss or netstat.
Diagnosing with ss
| Command | Shows |
|---|---|
ss -tan | All TCP sockets with state (numeric addresses) |
ss -tan state established | Only ESTABLISHED connections |
ss -tan state time-wait | wc -l | Count of TIME_WAIT sockets |
ss -tlnp | Listening servers with PID |
ss -s | Summary statistics per state |
TIME_WAIT Deep Dive
Why TIME_WAIT Exists and Why It Matters
COMMON ISSUEPurpose of TIME_WAIT (2 × MSL ≈ 60 seconds)
- Ensure the final ACK reaches the peer — if the peer's FIN is not ACK'd it will retransmit; TIME_WAIT allows us to re-ACK it.
- Prevent old segments from corrupting new connections — a new connection on the same 4-tuple must not see segments from the old connection (MSL = Maximum Segment Lifetime).
When TIME_WAIT Becomes a Problem
Each TIME_WAIT socket holds a 4-tuple (src-ip, src-port, dst-ip, dst-port). A server making many short outbound connections (HTTP/1.0 clients, aggressive connection teardown) can exhaust the ephemeral port range (~28K ports by default on Linux).
Solutions
- HTTP keep-alive / connection pooling — reuse connections, avoid teardown
net.ipv4.tcp_tw_reuse=1— allows reuse of TIME_WAIT connections for outbound; safe for clientsSO_REUSEADDR— allows server to bind to a port that has TIME_WAIT connections- Increase ephemeral port range:
net.ipv4.ip_local_port_range = 1024 65535
⚠️ Do NOT set tcp_tw_recycle — it was removed in Linux 4.12 because it breaks clients behind NAT (multiple clients appear to have same IP, causing packets to be dropped).
CLOSE_WAIT Accumulation: A Common Bug
CLOSE_WAIT Leak
BUG PATTERNIf ss shows many CLOSE_WAIT sockets, the peer has sent FIN but your application has not called close() on the socket. This is almost always a resource leak — your code received EOF but didn't clean up.
Root causes: forgetting to close the fd in error paths, connection not removed from a pool on EOF, async handler not calling close() after reading 0 bytes from recv().
Diagnosis: ss -tanp state close-wait to find which process, then check the source for missing close() calls after recv() == 0.
Half-Open Connections and Keepalive
Dead Peer Detection
RELIABILITYA half-open connection occurs when one side crashes without sending FIN (power loss, kernel panic, network cable unplugged). The surviving side believes the connection is ESTABLISHED but the peer is gone.
Without keepalive, this connection stays ESTABLISHED forever — wasting file descriptors and thread/process resources.
TCP Keepalive Settings (Linux)
| Sysctl | Default | Meaning |
|---|---|---|
tcp_keepalive_time | 7200s | Idle time before first probe |
tcp_keepalive_intvl | 75s | Interval between probes |
tcp_keepalive_probes | 9 | Probes before giving up |
Per-socket override (much faster than system defaults):
int idle = 10, intvl = 5, cnt = 3; setsockopt(fd, SOL_SOCKET, SO_KEEPALIVE, &(int){1}, sizeof(int)); setsockopt(fd, IPPROTO_TCP, TCP_KEEPIDLE, &idle, sizeof(idle)); setsockopt(fd, IPPROTO_TCP, TCP_KEEPINTVL, &intvl, sizeof(intvl)); setsockopt(fd, IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(cnt)); /* Now dead peer detected in 10 + 5×3 = 25 seconds */
TLS 1.3: Why It Matters
RFC 8446TLS 1.3 (2018) is a ground-up redesign of TLS 1.2. It removes legacy cruft (RSA key exchange, CBC mode ciphers, compression, renegotiation), cuts handshake latency from 2 RTT to 1 RTT, and mandates forward secrecy on every connection.
| Property | TLS 1.2 | TLS 1.3 |
|---|---|---|
| Handshake RTTs | 2 RTT | 1 RTT (0-RTT for resumption) |
| Key exchange | RSA (static) or ECDHE | ECDHE only (forward secrecy mandatory) |
| Cipher suites | 37+ (many weak) | 5 (all AEAD: AES-GCM, ChaCha20-Poly1305) |
| Certificate encryption | No | Yes (cert sent after key exchange) |
| Renegotiation | Supported (CVE source) | Removed |
| Compression | Optional (CRIME attack) | Removed |
TLS 1.3 Handshake: 1-RTT
Client Server
│ │
│── ClientHello ─────────────────────────────────────────────▶
│ • supported_versions: TLS 1.3 │
│ • key_share: client ECDHE public key (e.g. X25519) │
│ • supported_groups, signature_algorithms │
│ • psk_key_exchange_modes (for 0-RTT) │
│ │
│◀── ServerHello ─────────────────────────────────────────────
│ • key_share: server ECDHE public key │
│ • selected cipher suite │
│ │
│ [Both sides derive handshake traffic secrets via HKDF] │
│ │
│◀── {EncryptedExtensions} ───────────────────────────────────
│◀── {Certificate} ───────────────────────────────────────────
│◀── {CertificateVerify} ─────────────────────────────────────
│◀── {Finished} ──────────────────────────────────────────────
│ [server auth complete at this point] │
│ │
│── {Finished} ──────────────────────────────────────────────▶
│ [client auth if mutual TLS] │
│ │
│ [Both derive application traffic secrets] │
│ │
│══ {Application Data} ═══════════════════════════════════▶│◀│
│ (first app data can go with Finished — effectively 1 RTT) │
ECDHE: Forward Secrecy Explained
Ephemeral Diffie-Hellman (ECDHE)
FORWARD SECRECYIn old RSA key exchange, the client encrypted the session key with the server's public RSA key. If the server's private key was later stolen, all past recorded traffic could be decrypted.
ECDHE generates fresh key pairs per connection. Both sides exchange their ephemeral public keys; each derives the shared secret using their own private key and the peer's public key. The private key is never transmitted and is discarded after the handshake.
Result: Compromising the server's long-term certificate key cannot decrypt past sessions — each session's secret was derived from ephemeral keys that no longer exist.
Common curves in TLS 1.3: X25519 (preferred, fast, safe), P-256, P-384. X25519 is a modern curve with better performance and simpler implementation than NIST curves.
Certificate Chain Validation
- 1Receive cert chain — server sends its certificate and any intermediate CA certs. Leaf cert → intermediate → root CA.
- 2Verify signatures — each cert is signed by the one above it. Verify each signature up to a trusted root.
- 3Check trust anchor — root CA must be in the system trust store (
/etc/ssl/certs/on Linux). Browsers ship their own trust store. - 4Verify hostname (SNI) — leaf cert's Subject Alternative Names (SANs) must match the hostname being connected to.
- 5Check revocation — via CRL (Certificate Revocation List) or OCSP (Online Certificate Status Protocol). Browsers may use OCSP stapling to speed this up.
- 6Check validity period — cert must not be expired or not-yet-valid. Short-lived certs (90 days from Let's Encrypt) are best practice.
SNI and ALPN
SNI
SERVER NAME INDICATIONSNI allows a single server IP to host multiple TLS domains. The client sends the desired hostname in ClientHello before the server has selected a certificate — so the server can return the right cert.
In TLS 1.3, SNI is encrypted (via Encrypted Client Hello / ECH) to prevent observers from seeing which domain you're connecting to.
ALPN
APPLICATION LAYER PROTOCOL NEGOTIATIONALPN lets the client advertise which application protocols it supports (h2, http/1.1, h3) in ClientHello. The server picks one and includes it in ServerHello.
This is how a single port 443 server can serve both HTTP/1.1 and HTTP/2 connections without a separate port per protocol.
0-RTT Session Resumption
0-RTT (Early Data)
PERFORMANCE vs SECURITYAfter a 1-RTT handshake, the server sends a session ticket — a blob the client can present in the next ClientHello to skip the handshake entirely and send application data immediately.
Mechanism
Server encrypts a PSK (Pre-Shared Key) with its own ticket key and sends it in a NewSessionTicket message. On reconnect, the client sends this ticket + early data in the first message.
Security Trade-off: Replay Attacks
0-RTT data has no replay protection. An attacker who captures the first flight can replay it to a different server. Therefore:
- Never allow 0-RTT for non-idempotent operations (POST, DELETE, payments)
- Safe for: GET requests, read-only operations, connection warm-up
- Servers should use single-use tickets or replay caches to mitigate
Common TLS Mistakes
| Mistake | Risk | Fix |
|---|---|---|
Disabling cert verification (SSL_VERIFY_NONE) | MITM attacks, impersonation | Always verify in production; use SSL_VERIFY_PEER |
| Allowing TLS 1.0/1.1 | POODLE, BEAST, other protocol attacks | Set minimum to TLS 1.2; prefer TLS 1.3 only |
| Not checking hostname in SAN | Any cert from any CA accepted | Use SSL_set_hostflags + SSL_set1_host or verify via library |
| Hardcoded cipher suites with RC4/DES/3DES | Brute-forceable in hours | Use TLS_AES_128_GCM_SHA256 / TLS_CHACHA20_POLY1305_SHA256 |
| Ignoring cert expiry in automation | Outages when cert expires (famous ones every year) | Set up auto-renewal (certbot), alert at 30 days |
| Using self-signed certs in prod | Clients reject or users click through warnings | Use Let's Encrypt (free, 90-day, automatable) |
TCP Client (C)
#include <sys/socket.h> #include <netdb.h> #include <unistd.h> #include <string.h> #include <stdio.h> #include <errno.h> int tcp_connect(const char *host, const char *port) { struct addrinfo hints = { .ai_family = AF_UNSPEC, .ai_socktype = SOCK_STREAM }; struct addrinfo *res; int rc = getaddrinfo(host, port, &hints, &res); if (rc) { fprintf(stderr, "DNS: %s\n", gai_strerror(rc)); return -1; } int fd = -1; for (struct addrinfo *rp = res; rp; rp = rp->ai_next) { fd = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol); if (fd < 0) continue; if (connect(fd, rp->ai_addr, rp->ai_addrlen) == 0) break; close(fd); fd = -1; } freeaddrinfo(res); if (fd < 0) perror("connect"); return fd; } int main(void) { int fd = tcp_connect("httpbin.org", "80"); if (fd < 0) return 1; const char *req = "GET /get HTTP/1.1\r\nHost: httpbin.org\r\nConnection: close\r\n\r\n"; write(fd, req, strlen(req)); char buf[4096]; ssize_t n; while ((n = read(fd, buf, sizeof(buf))) > 0) fwrite(buf, 1, n, stdout); close(fd); }
TLS Client with OpenSSL
#include <openssl/ssl.h> #include <openssl/err.h> #include <sys/socket.h> #include <netdb.h> #include <unistd.h> #include <stdio.h> #include <string.h> /* Compile: gcc tls_client.c -lssl -lcrypto -o tls_client */ static int tcp_connect_fd(const char *host, const char *port); /* as above */ int main(int argc, char **argv) { const char *host = argc > 1 ? argv[1] : "example.com"; /* 1. Init OpenSSL */ SSL_library_init(); SSL_load_error_strings(); OpenSSL_add_all_algorithms(); /* 2. Create TLS context — prefer TLS 1.3 */ SSL_CTX *ctx = SSL_CTX_new(TLS_client_method()); SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION); /* 3. Load system CA bundle for cert verification */ SSL_CTX_set_default_verify_paths(ctx); SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL); /* 4. TCP connect */ int fd = tcp_connect_fd(host, "443"); if (fd < 0) { SSL_CTX_free(ctx); return 1; } /* 5. Wrap socket in SSL */ SSL *ssl = SSL_new(ctx); SSL_set_fd(ssl, fd); /* 6. Set SNI so server returns correct cert */ SSL_set_tlsext_host_name(ssl, host); /* 7. Set hostname for cert validation */ SSL_set_hostflags(ssl, X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS); SSL_set1_host(ssl, host); /* 8. TLS handshake */ int err = SSL_connect(ssl); if (err != 1) { ERR_print_errors_fp(stderr); goto cleanup; } /* 9. Print negotiated cipher and protocol */ printf("TLS version : %s\n", SSL_get_version(ssl)); printf("Cipher suite: %s\n", SSL_get_cipher(ssl)); /* 10. Print server cert info */ X509 *cert = SSL_get_peer_certificate(ssl); if (cert) { char buf[256]; X509_NAME_oneline(X509_get_subject_name(cert), buf, sizeof(buf)); printf("Cert subject: %s\n", buf); X509_NAME_oneline(X509_get_issuer_name(cert), buf, sizeof(buf)); printf("Cert issuer : %s\n", buf); X509_free(cert); } /* 11. Send HTTP request */ char req[512]; snprintf(req, sizeof(req), "GET / HTTP/1.1\r\nHost: %s\r\nConnection: close\r\n\r\n", host); SSL_write(ssl, req, strlen(req)); /* 12. Read response */ char rbuf[4096]; int n; while ((n = SSL_read(ssl, rbuf, sizeof(rbuf))) > 0) fwrite(rbuf, 1, n, stdout); cleanup: SSL_shutdown(ssl); SSL_free(ssl); close(fd); SSL_CTX_free(ctx); return 0; }
TCP Server with Concurrent Connections (pthreads)
#include <sys/socket.h> #include <netinet/in.h> #include <pthread.h> #include <unistd.h> #include <stdio.h> static void *handle_client(void *arg) { int fd = (int)(intptr_t)arg; pthread_detach(pthread_self()); /* auto-reclaim resources */ char buf[4096]; ssize_t n; while ((n = recv(fd, buf, sizeof(buf), 0)) > 0) { /* Echo back */ send(fd, buf, n, 0); } /* n == 0: peer closed; n < 0: error */ close(fd); return NULL; } int main(void) { int srv = socket(AF_INET, SOCK_STREAM, 0); setsockopt(srv, SOL_SOCKET, SO_REUSEADDR, &(int){1}, sizeof(int)); struct sockaddr_in a = {AF_INET, htons(8080), .sin_addr.s_addr=INADDR_ANY}; bind(srv, (struct sockaddr *)&a, sizeof(a)); listen(srv, 128); printf("Echo server :8080\n"); while (1) { int conn = accept(srv, NULL, NULL); if (conn < 0) { perror("accept"); continue; } pthread_t t; pthread_create(&t, NULL, handle_client, (void *)(intptr_t)conn); /* thread detached inside handle_client */ } }
⚠️ Thread-per-connection doesn't scale beyond a few thousand connections — each thread uses ~8 MB stack by default. For high concurrency, use epoll + event loop (covered in Phase 4 — I/O Multiplexing). This example is correct for understanding the basic model.
Error Handling Patterns
Handling EINTR, EAGAIN, Partial Reads/Writes
RELIABILITYEINTR — Interrupted System Call
Signal delivery can interrupt blocking I/O. Always restart on EINTR:
ssize_t safe_read(int fd, void *buf, size_t len) { ssize_t n; do { n = read(fd, buf, len); } while (n == -1 && errno == EINTR); return n; }
Partial Reads/Writes on TCP
TCP is a byte stream — send() may transfer fewer bytes than requested. Always loop:
ssize_t send_all(int fd, const void *buf, size_t len) { size_t sent = 0; while (sent < len) { ssize_t n = send(fd, (const char*)buf + sent, len - sent, MSG_NOSIGNAL); if (n <= 0) return n; sent += n; } return sent; } /* MSG_NOSIGNAL: don't raise SIGPIPE on broken pipe — return EPIPE instead */
🔬 Lab 1 — Wireshark: Observe DNS + TCP + TLS
TOOLS: Wireshark · curlGoal: Capture and annotate all three protocol phases for a single HTTPS request. See the latency budget with your own eyes.
dns or tcp.port == 443sudo systemd-resolve --flush-caches && curl -v https://example.com 2>&1 | head -40Expected findings: DNS: ~20–100 ms. TCP: 1 RTT. TLS 1.3: 1 RTT. Total before first byte ≈ DNS + 2 RTTs. TLS ClientHello and Certificate records are visible; application data is opaque (encrypted).
🔬 Lab 2 — TCP Echo Server in C
TOOLS: gcc · telnet · netcat · ssGoal: Build and test the echo server, then observe TCP state transitions in real time.
echo.c. Compile: gcc echo.c -lpthread -o echo./echo &. Check it's listening: ss -tlnp | grep 8080nc localhost 8080. Type lines — each should echo back. In another terminal: ss -tan | grep 8080. Observe ESTABLISHED state.ss -tan | grep 8080 multiple times. Observe TIME_WAIT appearing then disappearing after ~60s.kill %1. Try to restart immediately. Does it fail with "Address already in use"? Why? Now add SO_REUSEADDR if not present and retry.inet_ntop() on the sockaddr_in returned by accept().🔬 Lab 3 — TLS Client: Connect, Print Cert Chain & Cipher
TOOLS: gcc · OpenSSL · openssl CLIGoal: Build the OpenSSL TLS client, connect to a real server, and examine the full certificate chain.
sudo apt install libssl-dev (Debian/Ubuntu) or brew install openssl (macOS).tls_client.c. Add the tcp_connect_fd() implementation (from the TCP Client section). Compile: gcc tls_client.c -lssl -lcrypto -o tls_client./tls_client github.com. You should see TLS version (TLS 1.3), cipher suite (e.g. TLS_AES_128_GCM_SHA256), subject, and issuer.STACK_OF(X509) *chain = SSL_get_peer_cert_chain(ssl); for(int i=0; i<sk_X509_num(chain); i++) { ... }openssl s_client -connect github.com:443 -showcerts. Compare cert subjects, cipher, and protocol with your client's output.cloudflare.com). Then intentionally connect to a server with an expired/self-signed cert — observe the verification error. Handle it gracefully.SSL_SESSION_print_fp() to inspect the session ticket. Store it and present it on reconnect. Measure latency difference.🔬 Lab 4 — DNS Deep Dive with dig
TOOLS: dig · tcpdump · hostGoal: Walk the DNS resolution chain manually and understand TTLs, record types, and DNSSEC.
dig +trace github.com A. Observe: root nameservers → .com TLD → github.com authoritative. Note TTL at each level.dig github.com ANY. List A, AAAA, MX, NS, TXT records. What TXT records exist? (SPF? DKIM selectors?)dig nonexistent.github.com A. Note NXDOMAIN and the negative TTL in the SOA record's MINIMUM field.dig +dnssec cloudflare.com A. Do you see RRSIG records? What key tag is used?for i in $(seq 10); do dig +stats github.com A | grep "Query time"; done. First query is cold; subsequent should be fast (cached by resolver). Note the caching effect.Module Mastery Checklist
M01 COMPLETEYou have mastered this module when you can check off every item below without referring to notes.
DNS
- Explain the full recursive DNS resolution chain — stub resolver → recursive resolver → root → TLD → authoritative NS
- State the purpose of TTL and explain the two-phase TTL strategy for planned failovers (lower TTL before, restore after)
- Name at least 6 DNS record types and their use cases (A, AAAA, CNAME, MX, TXT, SRV)
- Explain why
getaddrinfo()must be used instead ofgethostbyname() - Describe how DNS cache poisoning works and name three defences (source port randomisation, DNSSEC, DoH/DoT)
- Use
dig +traceto walk the resolution chain; interpret the TTL values at each delegation level
TCP — Handshake & Options
- Draw the 3-way handshake with correct flag names (SYN, SYN-ACK, ACK) and state transitions on both sides
- Explain why ISNs must be randomly generated and the historical attack they prevent
- Name the 4 TCP options negotiated during the SYN exchange and what each enables
- Explain the difference between FIN (graceful) and RST (abortive) teardown; name 3 causes of RST
- Explain
SO_REUSEADDRandSO_REUSEPORT— when to use each and why - Write a TCP server socket setup in C:
socket() → setsockopt() → bind() → listen() → accept() - Handle partial writes correctly with a
send_all()loop; explain MSG_NOSIGNAL
TCP — State Machine
- Name all 11 TCP states and identify the 3 most commonly encountered in production (ESTABLISHED, TIME_WAIT, CLOSE_WAIT)
- Explain TIME_WAIT: why it exists (2 reasons), duration (2 × MSL), and production mitigations
- Diagnose a CLOSE_WAIT leak: what it means, how to find it with
ss, and what causes it in code - Configure per-socket TCP keepalive to detect dead peers within 25 seconds
- Use
ss -tanto count connections by state on a live server
TLS 1.3
- List 4 improvements TLS 1.3 made over TLS 1.2 (1-RTT, mandatory forward secrecy, fewer cipher suites, encrypted certs)
- Trace the TLS 1.3 1-RTT handshake: which messages are sent, in what order, and what each contains
- Explain forward secrecy: why ECDHE prevents decryption of past sessions even if the server key is compromised
- Explain 0-RTT resumption: how session tickets work, the replay attack risk, and which operations are safe/unsafe
- Explain SNI and ALPN: what problem each solves and when each is sent
- Write an OpenSSL TLS client that: sets
SSL_VERIFY_PEER, sets SNI, sets hostname for cert validation, prints the cipher suite and TLS version - Name 5 common TLS configuration mistakes and their consequences
Next module: M02 covers HTTP Internals — how HTTP/1.1, HTTP/2, and HTTP/3 use the TCP/TLS layer you just mastered. You'll understand pipelining, multiplexing, header compression (HPACK/QPACK), and QUIC's 0-RTT connection establishment.