{"id":"896b3df2-6485-49f6-8e63-1208d1ddeed6","task":"Deploy the OTel Collector as both a DaemonSet and a Deployment (gateway) on Kubernetes and understand the routing responsibilities of each tier","domain":"opentelemetry.io","steps":["Deploy an agent DaemonSet with the k8sattributes processor to enrich telemetry with pod, namespace, and node metadata before forwarding to the gateway","Configure the DaemonSet agent to receive OTLP from local pods via the service's localhost address and forward via OTLP to the gateway Deployment","Deploy the gateway Deployment with multiple replicas behind a Kubernetes Service; enable the loadbalancing exporter to route traces to tail-sampling Collector instances by trace ID","Assign the tail-sampling processor only to the gateway Deployment, not the agent, since tail sampling requires all spans of a trace to arrive at the same processor","Configure HPA (HorizontalPodAutoscaler) on the gateway Deployment based on CPU or received_spans metric from the Collector's internal Prometheus exporter"],"gotchas":["The DaemonSet agent must export to the gateway via a ClusterIP Service, not directly to backends; bypassing the gateway breaks tail sampling and central policy enforcement","The loadbalancing exporter uses trace ID hashing to assign traces to backends; adding or removing gateway replicas causes trace ID reshuffling and temporary incomplete traces during rollouts","Granting the DaemonSet agent the k8s:watch and k8s:list RBAC permissions for nodes, pods, and namespaces is required for the k8sattributes processor; missing permissions produce empty resource attributes"],"contributor":"waymark-seed","created":"2026-06-13T04:22:15.404Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/896b3df2-6485-49f6-8e63-1208d1ddeed6"}