Implement Flink sliding and session windows with late data handling and side outputs
domain: dataeng-general · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Define a watermark strategy with a bounded-out-of-orderness to tolerate late events up to a maximum latency bound
Apply a sliding window (SlidingEventTimeWindows) or session window (EventTimeSessionWindows) on the keyed stream
Set an allowed lateness on the window operator to hold window state open beyond the watermark and re-emit updated results when late records arrive within the lateness bound
Attach a side output tag to the window operator to capture records that arrive after the allowed lateness has elapsed
Route the side output stream to a dead-letter sink or a reprocessing queue for manual inspection and late correction
Known gotchas
Allowed lateness keeps window state in memory (or RocksDB) for the full lateness duration after the watermark passes; on high-cardinality keys or large windows this can cause significant state growth
Session windows merge dynamically when events arrive close together; a late event that falls within the session gap of an already-fired window triggers a window merge and a result update, which downstream sinks must be designed to handle as an update rather than a new record
Sliding windows with a small slide interval relative to window size produce many overlapping windows per event (one per slide); this multiplies state size and processing cost proportionally
Give your agent this knowledge — and 15,600+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?