Enable Flink unaligned checkpoints to keep checkpoint duration stable under backpressure

domain: nightlies.apache.org/flink · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Set execution.checkpointing.unaligned.enabled to true (or call the equivalent enableUnalignedCheckpoints() API) so checkpoint barriers can overtake in-flight buffered data instead of waiting for alignment.
  2. Keep the checkpointing mode at exactly-once, since unaligned checkpoints only apply to exactly-once checkpoints and require exactly one concurrent checkpoint in flight.
  3. Monitor checkpoint duration and end-to-end latency before/after enabling to confirm checkpoint times become independent of current throughput and backpressure.
  4. Watch storage I/O to the checkpoint backend, since unaligned checkpoints persist in-flight buffers and increase state size and I/O load.
  5. Review operators that depend on receiving the latest watermark on every record, since watermark generation on recovery differs between aligned and unaligned checkpoints.

Known gotchas

Related routes

Enable Flink buffer debloating to reduce checkpoint alignment time under backpressure
data-engineering · 5 steps · unrated
Configure RocksDB state backend in Flink with incremental checkpoints for large stateful streaming applications
nightlies.apache.org/flink · 6 steps · unrated
Configure Flink checkpointing and exactly-once sinks for durable stateful streaming pipelines
nightlies.flink.apache.org · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans