开门见山:Kafka Consumer,长时间 ( 甚至一直 ) 不 ACK 不会导致 rebalance!
But 有些小伙伴说,我的场景下 ( Java Client 默认配置下 ) 长时间不 ACK 会出现 rebalance。。。 实际上这是 Java Client 一个实现上的细节问题: 首先:在 Kafka Consumer 配置中,有个 max.poll.interval.ms 配置项,用来控制两次 poll 之间的超时时间;
The maximum delay between invocations of poll() when using consumer group management. This places an upper bound on the amount of time that the consumer can be idle before fetching more records. If poll() is not called before expiration of this timeout, then the consumer is considered failed and the group will rebalance in order to reassign the partitions to another member. For consumers using a non-null `group.instance.id` which reach this timeout, partitions will not be immediately reassigned. Instead, the consumer will stop sending heartbeats and partitions will be reassigned after expiration of `session.timeout.ms` . This mirrors the behavior of a static consumer which has shutdown.
其次:在 poll 的处理中,会调用 pollHeartbeat 更新 lastPoll,在另外独立线程则是执行 AbstractCoordinator.run 周期性检查 lastPoll 决定是否 leave group;
最后:在默认配置下,poll 处理最后会执行 maybeAutoCommitOffsetsAsync 提交 offset ( enable auto commit )。
因此,在静默模式下就会给人误区,以为长时间不 ACK 会触发 rebalance!