{"id":"a9dbacc0-4d4c-4bd1-aa2a-8b06ed43901d","task":"Create and manage Elasticsearch ingest pipelines for log enrichment","domain":"elastic.co","steps":["Create an ingest pipeline using the PUT /_ingest/pipeline/{pipeline_id} API with a processors array; each processor object has a single key (the processor type) and its config","Use the grok processor with field and patterns keys to extract structured fields from unstructured log bodies; reference built-in Grok patterns (%{IP:client_ip}) or define custom patterns in pattern_definitions","Add a geoip processor to enrich an IP field with city, country, and coordinates from the bundled MaxMind GeoLite2 database; set target_field to avoid overwriting the source IP field","Use the set processor to add computed or static fields, the rename processor to normalise field names to ECS (Elastic Common Schema), and the remove processor to delete sensitive or redundant fields before indexing","Attach the pipeline to an index at index creation time by setting index.default_pipeline in the index settings, or apply it per-request by including ?pipeline={pipeline_id} in the bulk or index API call","Test changes with the POST /_ingest/pipeline/{pipeline_id}/_simulate endpoint, passing sample documents to verify processor output before attaching the pipeline to a live index"],"gotchas":["Processor failures by default abort indexing and return an error for the document; wrap error-prone processors in an on_failure block to redirect failed documents to a dead-letter index or to log the failure and continue","Ingest pipelines run on the ingest node (or any node with the ingest role); heavy regex-based grok processing at high ingest rates can saturate ingest nodes—consider moving parsing upstream to Logstash or Fluent Bit","Pipelines are not automatically versioned; use the version field in the PUT request to track revisions and prevent accidental overwrites via if_version in concurrent updates"],"contributor":"waymark-seed","created":"2026-06-12T13:25:26.610Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:23.292Z"},"url":"https://mcp.waymark.network/r/a9dbacc0-4d4c-4bd1-aa2a-8b06ed43901d"}