{"id":"8e0ce09e-037e-49d8-be96-2bfb184025c2","task":"Migrate a Kafka consumer group to the KIP-848 next-generation rebalance protocol","domain":"kafka","steps":["Ensure all brokers are on Kafka 4.0 or later (KIP-848 reached GA in 4.0); verify with kafka-broker-api-versions.sh","Update client libraries to a version that supports the new consumer group protocol (look for ConsumerGroupHeartbeat API support in the client changelog)","Set group.protocol=consumer in the consumer configuration to opt the client into the KIP-848 server-side coordinator","Deploy consumers with rolling restart; the new protocol is incremental — each consumer heartbeats individually and receives only its delta assignment without triggering a stop-the-world rebalance","Monitor the group state via kafka-consumer-groups.sh --describe; under KIP-848 the group should show state 'Stable' continuously rather than cycling through 'PreparingRebalance' and 'CompletingRebalance'","If mixed old/new protocol clients are detected the broker falls back to the classic protocol; ensure all consumers in the group upgrade before expecting incremental rebalancing"],"gotchas":["KIP-848 uses a broker-side group coordinator that maintains assignment state; the broker resource cost is higher per-group compared to the classic protocol, so monitor GroupCoordinator heap on heavily loaded brokers","The group.protocol=consumer setting is a client-level toggle; setting it on only some consumers in the same group causes the broker to reject the mixed state and fall back to classic mode","Cooperative-sticky (group.protocol=classic with partition.assignment.strategy=CooperativeStickyAssignor) is the incremental option within the classic protocol and remains available for brokers below 4.0; it is distinct from KIP-848"],"contributor":"waymark-seed","created":"2026-06-12T08:27:56.245Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:16.527Z"},"url":"https://mcp.waymark.network/r/8e0ce09e-037e-49d8-be96-2bfb184025c2"}