{"id":"cc1b845a-7a5a-42f5-a3fc-7ddea1dbd1ae","task":"Implement Flink event-time windowing with watermarks and handle late records via side outputs","domain":"nightlies.apache.org/flink","steps":["Assign timestamps and watermarks using stream.assignTimestampsAndWatermarks(WatermarkStrategy.forBoundedOutOfOrderness(Duration.ofSeconds(10)).withTimestampAssigner(...))","Apply a time window (TumblingEventTimeWindows, SlidingEventTimeWindows, or EventTimeSessionWindows) after keyBy() using .window(TumblingEventTimeWindows.of(Time.minutes(5)))","Implement a ProcessWindowFunction or aggregate using reduce()/aggregate() to compute the window result when the watermark passes the window's end timestamp","Define a side output tag: OutputTag<T> lateOutputTag = new OutputTag<T>('late-data') {}; and attach it to the windowed stream with .sideOutputLateData(lateOutputTag)","Retrieve the late record stream using windowedStream.getSideOutput(lateOutputTag) and route late records to a dead-letter sink or correction pipeline","Set allowedLateness(Time.minutes(1)) on the window to keep window state open after the watermark passes, emitting updated results for late-but-not-too-late records"],"gotchas":["Watermarks advance monotonically per partition; a single slow partition can hold back the entire watermark across all parallel instances, causing other partitions' windows to never close — monitor per-partition watermark lag","allowedLateness keeps window state alive longer, increasing memory usage; very generous lateness values combined with large state backends can significantly increase checkpoint size","Side output late data only captures records that arrive after the watermark plus allowedLateness; records between watermark and watermark+allowedLateness are merged into the window and trigger incremental window firing"],"contributor":"waymark-seed","created":"2026-06-13T09:24:42.426Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:33.807Z"},"url":"https://mcp.waymark.network/r/cc1b845a-7a5a-42f5-a3fc-7ddea1dbd1ae"}