Configure Flink watermark strategies with bounded-out-of-orderness and route genuinely late records to a side output

domain: nightlies.flink.apache.org · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Assign a WatermarkStrategy using WatermarkStrategy.forBoundedOutOfOrderness(Duration.ofSeconds(10)) and chain .withTimestampAssigner() to extract the event timestamp field from each record
  2. On the windowed operator, set .allowedLateness(Time.seconds(5)) to accept records that arrive after the watermark but within a 5-second grace period — these re-trigger the window and emit an updated result
  3. Define a side output tag: OutputTag<MyEvent> lateTag = new OutputTag<>("late-events"){} and pass it to .sideOutputLateData(lateTag) on the windowed stream
  4. Collect the side output stream: DataStream<MyEvent> lateStream = windowedResult.getSideOutput(lateTag) and sink it to a monitoring topic or dead-letter store
  5. Tune the out-of-orderness bound to match observed source lag; too large a value increases end-to-end latency, too small causes correct records to be routed to the side output
  6. Monitor the numLateRecordsDropped metric in the Flink UI to confirm the side output is catching genuinely late data and the bound is appropriately calibrated

Known gotchas

Related routes

Implement Flink event-time windowing with watermarks and handle late records via side outputs
nightlies.apache.org/flink · 6 steps · unrated
Configure Spark Structured Streaming watermarking to handle late-arriving data and bound state size
spark.apache.org · 6 steps · unrated
Enable Flink buffer debloating to reduce checkpoint alignment time under backpressure
data-engineering · 5 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans