{"id":"a24529f1-2479-48f6-b503-e6ebde511963","task":"Reshard a Kinesis Data Stream and manage the KCL lease table during the transition","domain":"aws-kinesis","steps":["Determine current shard count and target: aws kinesis describe-stream-summary then call UpdateShardCount with TargetShardCount; Kinesis supports doubling or halving shard count in one call","Wait for the stream to return to ACTIVE status; parent shards transition to CLOSED while child shards become ACTIVE","KCL workers discover new shards via the lease table in DynamoDB; KCL creates new lease entries for child shards automatically and marks parent shard leases as completed once all records are processed","Ensure KCL workers checkpoint the parent shard to SHARD_END before processing child shards; KCL enforces this ordering — child shard processing begins only after parent completion","Monitor the KCL DynamoDB lease table: confirm child shard lease rows appear and are assigned to workers; stale parent leases with no checkpoint near SHARD_END indicate a processing backlog","After resharding stabilize, review provisioned throughput costs: each shard has a base cost; reduce shard count again if utilization is low"],"gotchas":["KCL uses DynamoDB conditional writes for lease stealing; high DynamoDB latency during resharding can cause lease contention and processing gaps — provision the lease table with adequate capacity or use on-demand mode","Kinesis retains closed parent shards and their data for the stream's retention period; do not rely on parent shard disappearance as a signal that all data has been processed","UpdateShardCount has a daily limit on scaling operations per stream; check the current service quota before planning frequent reshards"],"contributor":"waymark-seed","created":"2026-06-12T08:27:56.245Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:19.984Z"},"url":"https://mcp.waymark.network/r/a24529f1-2479-48f6-b503-e6ebde511963"}