```dockerfile # -- Stage 1: Build (fat image with full toolchain) -- FROM gcc:13 AS builder WORKDIR /src

Copy source and build system first (layer caching)#

COPY Makefile ./ COPY src/ ./src/

Build release binary (strip debug symbols)#

RUN make release CFLAGS=“-O2 -DNDEBUG” && strip bin/server

— Stage 2: Minimal runtime —#

FROM debian:bookworm-slim RUN apt-get update
&& apt-get install -y —no-install-recommends libpq5 ca-certificates
&& rm -rf /var/lib/apt/lists/*

Non-root user for security#

RUN useradd -r -u 1001 -s /sbin/nologin appuser WORKDIR /app COPY —from=builder /src/bin/server . USER appuser

EXPOSE 8080 HEALTHCHECK —interval=30s —timeout=3s
CMD curl -f http://localhost:8080/health || exit 1

ENTRYPOINT [”./server”]

    </div>
    <div class="ch-section-label">Concept Checklist</div>
    <table class="ch-problem-table">
      <thead><tr><th>✓</th><th>#</th><th>Concept</th><th>Category</th></tr></thead>
      <tbody>
        <tr data-key="ph6-c1"><td class="solved-cell"><div class="solved-check"></div></td><td>1</td><td>Monolith vs microservices: Conway's Law, bounded contexts, modular monolith first</td><td><span class="cat-badge">Architecture</span></td></tr>
        <tr data-key="ph6-c2"><td class="solved-cell"><div class="solved-check"></div></td><td>2</td><td>Strangler Fig: incremental migration via API gateway routing</td><td><span class="cat-badge">Architecture</span></td></tr>
        <tr data-key="ph6-c3"><td class="solved-cell"><div class="solved-check"></div></td><td>3</td><td>Sync vs async inter-service communication: tradeoffs, correlation ID pattern</td><td><span class="cat-badge">Communication</span></td></tr>
        <tr data-key="ph6-c4"><td class="solved-cell"><div class="solved-check"></div></td><td>4</td><td>API Gateway: routing, auth offload, rate limiting, BFF pattern</td><td><span class="cat-badge">Infra</span></td></tr>
        <tr data-key="ph6-c5"><td class="solved-cell"><div class="solved-check"></div></td><td>5</td><td>Service discovery: client-side (Consul) vs server-side vs DNS-based (K8s)</td><td><span class="cat-badge">Infra</span></td></tr>
        <tr data-key="ph6-c6"><td class="solved-cell"><div class="solved-check"></div></td><td>6</td><td>Circuit breaker: closed/open/half-open states, bulkhead pattern</td><td><span class="cat-badge">Reliability</span></td></tr>
        <tr data-key="ph6-c7"><td class="solved-cell"><div class="solved-check"></div></td><td>7</td><td>Docker multi-stage build for C/C++: builder → slim runtime, non-root user</td><td><span class="cat-badge">Docker</span></td></tr>
        <tr data-key="ph6-c8"><td class="solved-cell"><div class="solved-check"></div></td><td>8</td><td>Kubernetes: Pod, Deployment, Service, Ingress, liveness vs readiness probe</td><td><span class="cat-badge">K8s</span></td></tr>
        <tr data-key="ph6-c9"><td class="solved-cell"><div class="solved-check"></div></td><td>9</td><td>CI/CD: pipeline stages, blue-green deployment, canary traffic splitting</td><td><span class="cat-badge">CI/CD</span></td></tr>
        <tr data-key="ph6-c10"><td class="solved-cell"><div class="solved-check"></div></td><td>10</td><td>12-Factor App: config via env, stateless processes, stdout logs</td><td><span class="cat-badge">Best Practices</span></td></tr>
      </tbody>
    </table>
  </div>
</div>
<!-- ═══════════════════════════════════════════════════════════
     Ph7 — Observability & Hardening
     ═══════════════════════════════════════════════════════════ -->
<div class="ch-card" id="ph7" data-ch="ph7" style="--ch-accent: linear-gradient(90deg,#e11d48,#dc2626);">
  <div class="ch-header">
    <div class="ch-num">Ph7</div>
    <div class="ch-title-wrap">
      <div class="ch-title">Observability &amp; Hardening</div>
      <div class="ch-meta">
        <span class="ch-badge">Production</span>
        <span class="ch-badge">Requires Ph6</span>
        <a href="/learning/backend/m17-observability/" class="ch-badge notes-live">📄 M17 Notes</a>
      </div>
    </div>
    <span class="ch-chevron">▼</span>
  </div>
  <div class="ch-progress-row">
    <div class="ch-prog-bar-wrap"><div class="ch-prog-bar"></div></div>
    <span class="ch-prog-text">0/12 checked</span>
    <span class="ch-prog-pct">0%</span>
  </div>
  <div class="ch-body">
    <div class="ch-section-label">Key Concepts</div>
    <div class="be-concept-box">
      <ul>
        <li><strong>3 pillars of observability:</strong> logs (discrete events — what happened), metrics (aggregated numeric data — how many/how fast), traces (causal chains across services — why it was slow); each answers different questions; together give full system visibility</li>
        <li><strong>Structured logging:</strong> JSON lines format (one JSON object per line), mandatory fields (timestamp ISO-8601, level, service, trace_id, span_id, message), log levels (DEBUG verbose, INFO normal, WARN degraded, ERROR unexpected failure, FATAL unrecoverable), never log secrets or PII, use correlation/trace IDs to link logs across services</li>
        <li><strong>Metrics types:</strong> counter (monotonically increasing, e.g., <code>http_requests_total</code> — use <code>rate()</code>), gauge (point-in-time value, e.g., <code>memory_usage_bytes</code>, <code>active_connections</code>), histogram (bucketed distribution, e.g., <code>request_duration_seconds</code> — use <code>histogram_quantile()</code> for p99)</li>
        <li><strong>RED method</strong> (for services): Rate (requests/second), Errors (error rate %), Duration (latency percentiles p50/p95/p99); <strong>USE method</strong> (for resources): Utilization (% time busy), Saturation (queue depth, wait time), Errors (device error rate)</li>
        <li><strong>Prometheus:</strong> pull-based scraping (Prometheus polls <code>/metrics</code> endpoint on services), exposition format (<code># HELP</code>, <code># TYPE</code>, <code>metric_name{labels} value timestamp</code>), PromQL (<code>rate(http_requests_total[5m])</code>, <code>histogram_quantile(0.99, ...)</code>, <code>by(service)</code>), AlertManager for alerting rules and routing</li>
        <li><strong>Distributed tracing:</strong> trace (end-to-end request chain, unique trace_id), span (single operation within trace, span_id + parent_span_id), W3C <code>traceparent</code> header for cross-service propagation, OpenTelemetry SDK (language-agnostic instrumentation, OTLP export to Jaeger/Tempo/Zipkin)</li>
        <li><strong>Health check endpoints:</strong> <code>GET /health/live</code> — liveness probe (is process alive? if fails, Kubernetes restarts container), <code>GET /health/ready</code> — readiness probe (is service ready to serve traffic? if fails, removed from Service endpoints); startup probe for slow-starting containers</li>
        <li><strong>Rate limiting algorithms:</strong> token bucket (bucket refills at rate r, allow bursts up to capacity b — bursty traffic ok), sliding window log (store timestamps of all requests, exact but memory O(requests)), sliding window counter (approximate, memory O(1), compromise); implement at API gateway (global) and per-service (defense in depth)</li>
        <li><strong>OWASP Top 10 for backends:</strong> SQL injection (parameterized queries only — never string concat), command injection (avoid shell=True / system(), use execv), SSRF (Server-Side Request Forgery — allowlist outbound URLs), broken access control (check authorization on every request, not just auth), security misconfiguration (disable debug endpoints in prod, no default credentials), insecure deserialization (validate and sanitize all deserialized input)</li>
        <li><strong>Input validation:</strong> allowlist over denylist (define what is allowed, reject everything else), validate at trust boundaries only (never trust client input), size limits (prevent DoS via large payloads — max body size), type checking, sanitize before SQL/shell/HTML context</li>
        <li><strong>Secrets management:</strong> never in source code or Docker images (scan with truffleHog/gitleaks), environment variables (basic, visible in <code>/proc/PID/environ</code> — acceptable for containers), HashiCorp Vault (dynamic secrets with TTL + auto-rotation, audit log, fine-grained policies), AWS Secrets Manager / GCP Secret Manager; secret rotation strategy</li>
        <li><strong>Graceful shutdown:</strong> catch SIGTERM (Kubernetes sends this before SIGKILL after <code>terminationGracePeriodSeconds</code>), stop accepting new connections (close listen socket or remove from load balancer), drain in-flight requests (atomic counter), close DB connection pool, deregister from service discovery, log completion; target: shutdown in &lt; terminationGracePeriodSeconds (default 30s)</li>
      </ul>
    </div>
    <div class="ch-section-label">Technologies &amp; Tools</div>
    <div class="be-tools">
      <span class="be-tool-badge">Prometheus</span>
      <span class="be-tool-badge">Grafana</span>
      <span class="be-tool-badge">Jaeger</span>
      <span class="be-tool-badge">OpenTelemetry</span>
      <span class="be-tool-badge">HashiCorp Vault</span>
      <span class="be-tool-badge">AlertManager</span>
    </div>
    <button class="code-toggle-btn"><span class="caret">▸</span> Show C Example</button>
    <div class="code-block-wrap">
```c
/* Graceful shutdown via SIGTERM — C implementation */
#include <signal.h>
#include <stdatomic.h>
#include <stdio.h>
#include <unistd.h>

static atomic_int  in_flight        = 0;
static atomic_bool shutdown_req     = false;

static void handle_sigterm(int sig) {
    (void)sig;
    atomic_store(&shutdown_req, true);
}

void register_signals(void) {
    struct sigaction sa = { .sa_handler = handle_sigterm };
    sigemptyset(&sa.sa_mask);
    sigaction(SIGTERM, &sa, NULL);
    sigaction(SIGINT,  &sa, NULL);   /* also handle Ctrl-C */
}

/* Called at start of each request handler */
void request_begin(void) { atomic_fetch_add(&in_flight, 1); }

/* Called at end of each request handler */
void request_end(void)   { atomic_fetch_sub(&in_flight, 1); }

int main(void) {
    register_signals();
    /* ... start server, accept connections ... */

    /* Main loop — stop accepting when shutdown requested */
    while (!atomic_load(&shutdown_req)) {
        /* accept() new connections */
    }

    /* Drain: wait for all in-flight requests to complete */
    fprintf(stderr, "[shutdown] draining %d in-flight requests\n",
            atomic_load(&in_flight));
    while (atomic_load(&in_flight) > 0)
        usleep(5000);   /* poll every 5ms */

    /* Close DB pools, deregister from service discovery */
    fprintf(stderr, "[shutdown] clean exit\n");
    return 0;
}

</div>
<div class="ch-section-label">Concept Checklist</div>
<table class="ch-problem-table">
  <thead><tr><th>✓</th><th>#</th><th>Concept</th><th>Category</th></tr></thead>
  <tbody>
    <tr data-key="ph7-c1"><td class="solved-cell"><div class="solved-check"></div></td><td>1</td><td>3 pillars: logs (events), metrics (aggregates), traces (causal chains) — what each answers</td><td><span class="cat-badge">Observability</span></td></tr>
    <tr data-key="ph7-c2"><td class="solved-cell"><div class="solved-check"></div></td><td>2</td><td>Structured logging: JSON lines, mandatory fields, log levels, trace_id correlation</td><td><span class="cat-badge">Observability</span></td></tr>
    <tr data-key="ph7-c3"><td class="solved-cell"><div class="solved-check"></div></td><td>3</td><td>Metric types: counter (rate()), gauge, histogram (histogram_quantile p99)</td><td><span class="cat-badge">Metrics</span></td></tr>
    <tr data-key="ph7-c4"><td class="solved-cell"><div class="solved-check"></div></td><td>4</td><td>RED method (Rate, Errors, Duration) and USE method (Utilization, Saturation, Errors)</td><td><span class="cat-badge">Metrics</span></td></tr>
    <tr data-key="ph7-c5"><td class="solved-cell"><div class="solved-check"></div></td><td>5</td><td>Prometheus: pull-based scraping, exposition format, PromQL, AlertManager</td><td><span class="cat-badge">Metrics</span></td></tr>
    <tr data-key="ph7-c6"><td class="solved-cell"><div class="solved-check"></div></td><td>6</td><td>Distributed tracing: trace/span model, W3C traceparent header, OpenTelemetry</td><td><span class="cat-badge">Tracing</span></td></tr>
    <tr data-key="ph7-c7"><td class="solved-cell"><div class="solved-check"></div></td><td>7</td><td>Health checks: liveness (restart) vs readiness (remove from LB) vs startup probe</td><td><span class="cat-badge">Reliability</span></td></tr>
    <tr data-key="ph7-c8"><td class="solved-cell"><div class="solved-check"></div></td><td>8</td><td>Rate limiting: token bucket, sliding window log, sliding window counter</td><td><span class="cat-badge">Performance</span></td></tr>
    <tr data-key="ph7-c9"><td class="solved-cell"><div class="solved-check"></div></td><td>9</td><td>OWASP Top 10: SQL injection, SSRF, broken access control, security misconfiguration</td><td><span class="cat-badge">Security</span></td></tr>
    <tr data-key="ph7-c10"><td class="solved-cell"><div class="solved-check"></div></td><td>10</td><td>Input validation: allowlist, trust boundaries, size limits, sanitization</td><td><span class="cat-badge">Security</span></td></tr>
    <tr data-key="ph7-c11"><td class="solved-cell"><div class="solved-check"></div></td><td>11</td><td>Secrets management: Vault dynamic secrets, never in code/images, rotation</td><td><span class="cat-badge">Security</span></td></tr>
    <tr data-key="ph7-c12"><td class="solved-cell"><div class="solved-check"></div></td><td>12</td><td>Graceful shutdown: SIGTERM handler, drain in-flight, close pools, deregister</td><td><span class="cat-badge">Reliability</span></td></tr>
  </tbody>
</table>