Manage Datadog log pipelines and processors via the API

Steps

1. Authenticate API calls with the headers DD-API-KEY and DD-APPLICATION-KEY (application key must have the logs_write_pipelines scope)
2. Create a pipeline with POST /api/v1/logs/config/pipelines; provide a name, filter.query string (e.g., source:nginx) to scope which logs enter the pipeline, and an is_enabled flag
3. Add processors inside the pipeline by including a processors array in the create or update body; each processor object requires a type (e.g., grok-parser, date-remapper, attribute-remapper, string-builder-processor), a name, and type-specific config fields
4. Update an existing pipeline (to add, reorder, or modify processors) with PUT /api/v1/logs/config/pipelines/{pipelineId}; this replaces the entire pipeline definition, so always GET the current state first and merge your changes
5. Control pipeline execution order with PUT /api/v1/logs/config/pipeline-order; the body is {"pipeline_ids": ["id1", "id2"]} in the desired evaluation order—pipelines are evaluated top-to-bottom and the first matching pipeline's processors run
6. Manage pipelines as code using the Datadog Terraform provider (resource datadog_logs_custom_pipeline) or Pulumi to version-control and review pipeline changes through normal CI/CD workflows