A DaemonSet in Kubernetes is a specialized type of controller that ensures a specific pod runs on every node (or a subset of nodes) in a Kubernetes cluster. This is useful when you need to deploy background system-level services like log collectors, monitoring agents, or networking plugins that must be present across the entire cluster. Once a node joins the cluster, the DaemonSet automatically schedules the designated pod on it, and if a node is removed, the pod is cleaned up.
Why DaemonSets Are Important
DaemonSets help manage cluster-wide services without manual intervention. For example, if you’re running a monitoring agent like Prometheus Node Exporter or a log collection tool like Fluentd, it needs to operate on every node to gather data. Instead of individually scheduling pods on each node, a DaemonSet automates the process. This not only saves time but ensures consistency and fault-tolerance across the environment.
How DaemonSets Work in Kubernetes
When a DaemonSet is created, the Kubernetes scheduler places one copy of the specified pod on each node that matches the configuration. It keeps watching for new nodes being added, and automatically spins up the same pod on them. If a node is deleted or tainted to become unschedulable, the DaemonSet controller ensures the pod is either stopped or moved based on your policy.
Common Use Cases for DaemonSets
DaemonSets are typically used for system-level tasks that must run on all nodes. These include:
- Log shipping agents like Fluentd or Filebeat
- Metrics collectors like Prometheus Node Exporter
- Security monitoring tools such as Falco
- Storage daemons or node-level volume managers
- Network policy enforcers like Calico or Cilium
They provide a way to enforce consistent runtime services on every node without needing manual deployment to each one.
Creating a DaemonSet: Basic Example
Here’s a simple example of a DaemonSet YAML manifest that deploys a pod with a basic container image on all nodes:
yaml
CopyEdit
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: example-daemonset
spec:
selector:
matchLabels:
name: example-pod
template:
metadata:
labels:
name: example-pod
spec:
containers:
– name: example-container
image: nginx
This YAML file tells Kubernetes to run one nginx pod on every node in the cluster.
Controlling Which Nodes Run DaemonSet Pods
You can use node selectors, node affinity, or taints and tolerations to control which nodes should run the DaemonSet’s pods. This is helpful when you don’t want the pod to be deployed cluster-wide, but only on specific types of nodes (e.g., GPU-enabled, SSD-backed, or nodes in a specific availability zone).
Updating a DaemonSet
Like Deployments, DaemonSets support rolling updates, so you can upgrade the container image or configurations without downtime. However, rolling updates must be explicitly configured. If not configured, Kubernetes deletes and recreates pods, which may cause temporary service gaps.
How DaemonSets Differ from Deployments
While both DaemonSets and Deployments manage pods, they serve different purposes. A Deployment is used to maintain a desired number of pod replicas across the cluster for application scalability. In contrast, a DaemonSet ensures exactly one pod runs on each (or specific) node. Deployments are about scaling horizontally; DaemonSets are about ensuring presence across infrastructure.
Monitoring and Troubleshooting DaemonSets
Like other Kubernetes objects, DaemonSets can be monitored with standard tools like kubectl. You can check the status with:
bash
CopyEdit
kubectl get daemonsets
Or inspect more deeply using:
bash
CopyEdit
kubectl describe daemonset <name>
If pods aren’t running on expected nodes, verify node labels, taints, and scheduling rules. Also ensure your cluster has available resources for scheduling.
Cleaning Up a DaemonSet
To delete a DaemonSet and its pods, use:
bash
CopyEdit
kubectl delete daemonset <name>
This command removes the controller and all the pods it created across the nodes. If you want to keep the pods but remove the DaemonSet object, you can delete it with –cascade=orphan.
Real-World Example: Fluent Bit DaemonSet
Suppose you want to deploy Fluent Bit as a logging agent on all nodes. You can use a DaemonSet to ensure logs are collected from each node and sent to a central location like Elasticsearch or CloudWatch. This guarantees consistent log visibility across your entire cluster without manual deployment on individual nodes.
Security Considerations with DaemonSets
DaemonSets often run pods with elevated privileges because they interact with system-level components. You must secure them properly by:
- Using PodSecurityPolicies (deprecated) or PodSecurity Admission rules
- Setting read-only file systems
- Running containers as non-root users where possible
- Scanning images for vulnerabilities
A compromised DaemonSet pod could potentially affect every node it runs on, so follow the principle of least privilege.
Best Practices When Using DaemonSets
- Use labels and selectors carefully to avoid accidental scheduling
- Monitor resource usage since each pod runs on every node
- Prefer lightweight containers to minimize overhead
- Validate configuration changes before rolling them out
- Combine with Helm or Kustomize for easier deployment in CI/CD
Following best practices helps maintain stability, security, and performance across your Kubernetes environment.
DaemonSet Alternatives and When Not to Use It
If you don’t need a pod on every node, consider using a Deployment with node affinity or a Job for one-off tasks. If pods should only run on certain node types, you might combine Deployments with node selectors. DaemonSets are ideal for always-on background services—not for general-purpose application scaling.
Conclusion
DaemonSets are a powerful Kubernetes feature designed for managing background pods that need to run on every node. They simplify the deployment of cluster-wide agents and system services, improving consistency, observability, and control. By understanding how they work and applying best practices, you can ensure your Kubernetes infrastructure remains efficient and reliable at scale.
