Back to Kubernetes Course

Why Deployments Aren't Enough for Stateful Apps

A Deployment treats its Pods as interchangeable — any replica can be scaled up, down, or replaced, with a randomly generated name and no guaranteed identity. This is perfect for stateless apps, but breaks down for applications like databases and message queues that need:

• A stable, predictable network identity for each replica (e.g., mongo-0 should always be mongo-0, not a new random name after every restart).
• Stable, dedicated storage per replica, that follows that specific replica even if it's rescheduled to a different node.
• Ordered, sequential startup and shutdown (e.g., a database primary must be up before replicas join).

StatefulSet is the controller designed for exactly these requirements.

---

StatefulSet Example

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: mongo-headless
  replicas: 3
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
        - name: mongo
          image: mongo:7.0
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-data
              mountPath: /data/db
  volumeClaimTemplates:
    - metadata:
        name: mongo-data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi

• volumeClaimTemplates is the key difference from a Deployment: instead of every replica sharing one PVC, Kubernetes creates a separate PVC for each replica (mongo-data-mongo-0, mongo-data-mongo-1, mongo-data-mongo-2), and that PVC follows its Pod even across rescheduling.
• Pods get predictable, stable names: mongo-0, mongo-1, mongo-2 — never randomly suffixed like a Deployment's Pods.
• Pods are created and scaled in order (mongo-0 before mongo-1 before mongo-2) and terminated in reverse order by default — important for clustered databases with leader/follower semantics.

---

Headless Services

StatefulSets are almost always paired with a headless Service (clusterIP: None), which doesn't load-balance — instead, DNS lookups return the individual IPs of every matching Pod, so each replica can be addressed directly:

apiVersion: v1
kind: Service
metadata:
  name: mongo-headless
spec:
  clusterIP: None
  selector:
    app: mongo
  ports:
    - port: 27017

Each Pod also gets its own stable DNS name: mongo-0.mongo-headless.default.svc.cluster.local — essential for clustered databases where nodes need to address each other individually, not through a single load-balanced endpoint.

kubectl get statefulset
kubectl get pods -l app=mongo -w    # Watch ordered creation
kubectl scale statefulset mongo --replicas=5

---

DaemonSets

A DaemonSet ensures that exactly one copy of a Pod runs on every node (or a chosen subset of nodes) — the opposite goal from a Deployment, which doesn't care which nodes Pods land on, just how many total replicas exist.

Common DaemonSet use cases:

• Log collection agents (e.g., Fluentd, Filebeat) that need to run on every node to collect logs from all Pods there.
• Monitoring agents (e.g., Prometheus Node Exporter, Datadog Agent) that report node-level metrics.
• Network plugins (many CNI implementations run as a DaemonSet themselves).
• Storage daemons that need direct access to node-local disks.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      containers:
        - name: node-exporter
          image: prom/node-exporter:latest
          ports:
            - containerPort: 9100

• No replicas field — a DaemonSet's replica count is implicitly "one per matching node."
• As new nodes join the cluster, the DaemonSet automatically schedules a Pod onto them; as nodes are removed, their Pods are garbage collected.
• spec.template.spec.nodeSelector or affinity rules can restrict a DaemonSet to a subset of nodes (e.g., only GPU nodes).

kubectl get daemonset -A
kubectl rollout status daemonset/node-exporter