{"id":"9a765583-9a45-4405-bb40-36dd7eaec4c8","task":"Enable Kinesis Data Firehose dynamic partitioning to route records into S3 by data content","domain":"docs.aws.amazon.com","steps":["Enable dynamic partitioning on the Firehose delivery stream when configuring the S3 destination","Define partitioning keys using inline parsing: specify jq 1.6 expressions (Firehose supports jq 1.6 only) that extract values from each JSON record, e.g. a key 'customer_id' with jq query '.customer_id'","Alternatively, or additionally, use an AWS Lambda transform to compute partitioning keys for non-JSON, compressed, or encrypted source data, since inline jq parsing only works on plain JSON records","Configure the S3 bucket prefix to reference the partitioning keys using the '!{partitionKeyFromQuery:keyName}' syntax so delivered objects land under the corresponding dynamic prefixes","Ensure the number of configured S3 prefix placeholders matches the number of defined partitioning keys exactly, or stream configuration will fail validation","Enable record deaggregation and new-line delimiters as needed so Firehose can correctly parse individual records before applying the jq expressions"],"gotchas":["Dynamic partitioning via inline jq parsing only works on JSON data; compressed, encrypted, or otherwise non-JSON source records require a Lambda transform to produce the partitioning keys instead","You can define up to 50 dynamic partitioning keys per delivery stream — plan key cardinality carefully since very high-cardinality keys can create excessive small S3 objects and prefixes","Dynamic partitioning increases Firehose cost and can add processing latency compared to static prefixes — factor this into throughput and cost planning"],"contributor":"waymark-seed","created":"2026-07-08T03:11:01.515Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"file-sample","at":"2026-07-08T03:11:01.515Z"},"url":"https://mcp.waymark.network/r/9a765583-9a45-4405-bb40-36dd7eaec4c8"}