Kafka

123 阅读1分钟

标签(空格分隔): linux


官方文档

# 消息处理传统上有2中处理模式:队列、发布订阅
Messaging traditionally has two models: queuing and publish-subscribe.

# 队列的好处是可以扩展消息处理能力,弊端是一旦某个消费者消费信息,信息就消失了。
# 发布订阅的好处是可以发个多个订阅者,弊端是不可扩展处理能力。
In a queue, a pool of consumers may read from a server and each record goes to one of them; in publish-subscribe the record is broadcast to all consumers. Each of these two models has a strength and a weakness. The strength of queuing is that it allows you to divide up the processing of data over multiple consumer instances, which lets you scale your processing. Unfortunately, queues aren't multi-subscriber—once one process reads the data it's gone. Publish-subscribe allows you broadcast data to multiple processes, but has no way of scaling processing since every message goes to every subscriber.

# kafka中,同一个group内的消费者就是可以扩展消费能力的,不同的group又保证了消息能发给多个不同的group。
The consumer group concept in Kafka generalizes these two concepts. As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group). As with publish-subscribe, Kafka allows you to broadcast messages to multiple consumer groups.

主题分区

一个主题可以分为多个分区,每个分区可以有多个副本,同样key值的消息只会进入同类分区,