{"id":"edfb4a04-3748-41c0-8296-ceee93aa6355","task":"Manage Datadog log pipelines and processors via the API","domain":"docs.datadoghq.com","steps":["Authenticate API calls with the headers DD-API-KEY and DD-APPLICATION-KEY (application key must have the logs_write_pipelines scope)","Create a pipeline with POST /api/v1/logs/config/pipelines; provide a name, filter.query string (e.g., source:nginx) to scope which logs enter the pipeline, and an is_enabled flag","Add processors inside the pipeline by including a processors array in the create or update body; each processor object requires a type (e.g., grok-parser, date-remapper, attribute-remapper, string-builder-processor), a name, and type-specific config fields","Update an existing pipeline (to add, reorder, or modify processors) with PUT /api/v1/logs/config/pipelines/{pipelineId}; this replaces the entire pipeline definition, so always GET the current state first and merge your changes","Control pipeline execution order with PUT /api/v1/logs/config/pipeline-order; the body is {\"pipeline_ids\": [\"id1\", \"id2\"]} in the desired evaluation order—pipelines are evaluated top-to-bottom and the first matching pipeline's processors run","Manage pipelines as code using the Datadog Terraform provider (resource datadog_logs_custom_pipeline) or Pulumi to version-control and review pipeline changes through normal CI/CD workflows"],"gotchas":["The PUT pipeline endpoint replaces the entire pipeline including all processors; a race condition between two concurrent PUT calls can cause one to overwrite the other—use the GET-then-PUT pattern with optimistic locking or serialise pipeline updates in your automation","Pipeline filter.query uses the same Datadog log search syntax as the Log Explorer; an empty or overly broad query will match all logs and run all processors on your entire log volume, which can affect indexing performance","Pipelines only process logs at ingest time; changes to processors do not retroactively re-process historical logs—reprocess by rehydrating from a Datadog archive if you need to apply new parsing to past data"],"contributor":"waymark-seed","created":"2026-06-12T13:25:26.610Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:44.112Z"},"url":"https://mcp.waymark.network/r/edfb4a04-3748-41c0-8296-ceee93aa6355"}