本文已参与「新人创作礼」活动，一起开启掘金创作之路。
Apache RocketMQ是一个分布式消息传递和流媒体平台，具有低延迟、高性能和可靠性、万亿级容量和灵活的可伸缩性。它由四部分组成:name servers, brokers, producers and consumers。它们中的每一个都可以水平扩展。
NameServer Cluster
NameServer提供轻量级的服务发现和路由。每个NameServer记录完整的路由信息，提供相应的读写服务，支持快速存储扩展。 NameServer用于协调分布式系统的各个组件，协调主要通过管理主题路由信息来实现
- Broker管理，NameServer从Broker集群接受注册，并提供心跳机制来检查Broker是否处于活动状态。Broker定期更新保存在每个NameServer中的元数据
- 路由管理，每个NameServer将保存关于Broker集群和客户端查询的队列信息的整个路由信息。NameServer为客户端提供服务，包括提供最新路由信息的生产者、消费者和命令行客户端。
启动 brokers 和 clients 需提供NameServer的服务器地址方式
- Programmatic Way：namesrvAddr=name-server-ip1:port;name-server-ip2:port
- Java Options：rocketmq.namesrv.addr
- Environment Variable：NAMESRV_ADDR
- HTTP Endpoint：
优先级
Programmatic Way > Java Options > Environment Variable > HTTP Endpoint
Broker Cluster
Broker通过提供轻量级TOPIC和QUEUE机制来处理消息存储。它们支持推拉模型，包含容错机制(2个或3个副本)，并提供了强大的峰值填充和按原始时间顺序积累数千亿条消息的能力。此外，Broker提供灾难恢复、丰富的度量统计数据和警报机制，这些都是传统消息传递系统所缺乏的。
代理服务器负责消息存储和传递、消息查询、HA保证等
- Remoting Module, the entry of broker, handles the requests from clients
- Client Manager, manages the clients (Producer/Consumer) and maintains topic subscription of consumer.
- Store Service, provides simple APIs to store or query message in physical disk.
- HA Service, provides data sync feature between master broker and slave broker.
- Index Service, builds index for messages by specified key and provides quick message query.
Best Practice
- Broker Role:
- ASYNC_MASTER（异步复制）
- SYNC_MASTER（同步复制）
- SLAVE（启动定时任务同步）
- FlushDiskType:
- ASYNC_FLUSH：异步刷盘。建议使用
- SYNC_FLUSH：同步刷盘。会导致太多性能损失
Broker configuration
| Property Name | Default value | Details |
|---|---|---|
| listenPort | 10911 | listen port for client |
| namesrvAddr | null | name server address |
| brokerIP1 | InetAddress for network interface | Should be configured if having multiple addresses |
| brokerName | null | broker name |
| brokerClusterName | DefaultCluster | this broker belongs to which cluster |
| brokerId | 0 | broker id, 0 means master, positive integers mean slave |
| storePathCommitLog | $HOME/store/commitlog/ | file path for commit log |
| storePathConsumerQueue | $HOME/store/consumequeue/ | file path for consume queue |
| mapedFileSizeCommitLog | 1024 * 1024 * 1024(1G) | mapped file size for commit log |
| deleteWhen | 04 | When to delete the commitlog which is out of the reserve time |
| fileReserverdTime | 72 | The number of hours to keep a commitlog before deleting it |
| brokerRole | ASYNC_MASTER | SYNC_MASTER/ASYNC_MASTER/SLVAE |
| flushDiskType | ASYNC_FLUSH | {SYNC_FLUSH/ASYNC_FLUSH}. Broker of SYNC_FLUSH mode flushes each message onto disk before acknowledging producer. Broker of ASYNC_FLUSH mode, on the other hand, takes advantage of group-committing, achieving better performance. |
Producter Cluster
生产者支持分布式部署。分布式生产者通过多种负载平衡模式向代理集群发送消息。发送进程支持快速故障和低延迟。
Best Practice
- SendResult.SendStatus
- FLUSH_DISK_TIMEOUT：如果Broker设置MessageStoreConfig的FlushDiskType=SYNC_FLUSH，并且Broker没有完成MessageStoreConfig的syncFlushTimeout内的磁盘刷新(默认情况下是5秒)，您将得到此状态
- FLUSH_SLAVE_TIMEOUT：如果代理的角色是SYNC_MASTER(默认情况下是ASYNC_MASTER)，并且slave没有在MessageStoreConfig的syncFlushTimeout内完成与maser的同步，那么您将获得此状态。
- SLAVE_NOT_AVAILABLE：如果代理的角色是SYNC_MASTER，但是没有配置salve，那么您将获得此状态。
- SEND_OK：并不意味着它是可靠的。为了确保不会丢失任何消息，还应该启用SYNC_MASTER或SYNC_FLUSH。
- Duplication or Missing
- Timeout：发送消息到Broker并等待响应结果，超时（默认3s）抛出异常RemotingTimeoutException。我们不建议等待时间过短，因为代理需要一些时间来刷新磁盘或与从服务器同步
- Message Size：建议消息的大小不超过512K
- Async Sending：默认send(msg)将阻塞，直到返回响应。因此，如果您关心性能，我们建议您使用send(msg, callback)，它将以异步的方式工作。
Consumer Cluster
使用者还支持推和拉模型中的分布式部署。它还支持集群消费和消息广播。它提供实时消息订阅机制，可以满足大多数用户的需求。RocketMQ的网站为感兴趣的用户提供了一个简单的快速入门指南。
Best Practice
- MessageListener
- Orderly：不建议抛出异常，您可以返回ConsumeOrderlyStatus。SUSPEND_CURRENT_QUEUE_A_MOMENT代替
- Concurrently：不建议抛出异常，您可以返回consumereconcurrentlystatus。RECONSUME_LATER代替
- Consume Status
- RECONSUME_LATER：MessageListenerConcurrently
- SUSPEND_CURRENT_QUEUE_A_MOMENT：MessageListenerOrderly
- ConsumeFromWhere
- CONSUME_FROM_LAST_OFFSET：忽略历史消息
- CONSUME_FROM_FIRST_OFFSET：使用代理中存在的所有消息
- CONSUME_FROM_TIMESTAMP：使用指定时间戳之后生成的消息
- Duplication：不可避免，自行处理
CLI Admin Tool
RocketMQ提供了一个CLI(命令行界面)管理工具带，用于查询、管理和诊断各种问题。
Configuration
Broker configuration
| Property Name | Default value | Details |
|---|---|---|
| listenPort | 10911 | listen port for client |
| namesrvAddr | null | name server address |
| brokerIP1 | InetAddress for network interface | Should be configured if having multiple addresses |
| brokerName | null | broker name |
| brokerClusterName | DefaultCluster | this broker belongs to which cluster |
| brokerId | 0 | broker id, 0 means master, positive integers mean slave |
| storePathCommitLog | $HOME/store/commitlog/ | file path for commit log |
| storePathConsumerQueue | $HOME/store/consumequeue/ | file path for consume queue |
| mapedFileSizeCommitLog | 1024 * 1024 * 1024(1G) | mapped file size for commit log |
| deleteWhen | 04 | When to delete the commitlog which is out of the reserve time |
| fileReserverdTime | 72 | The number of hours to keep a commitlog before deleting it |
| brokerRole | ASYNC_MASTER | SYNC_MASTER/ASYNC_MASTER/SLVAE |
| flushDiskType | ASYNC_FLUSH | {SYNC_FLUSH/ASYNC_FLUSH}. Broker of SYNC_FLUSH mode flushes each message onto disk before acknowledging producer. Broker of ASYNC_FLUSH mode, on the other hand, takes advantage of group-committing, achieving better performance. |
topic参数
perm（100读权限 , 10写权限 , 6是110读写权限）
部署
Name Server
> cd /data/rocketmq-all-4.5.1-bin-release
# startup
> nohup sh bin/mqnamesrv -n localhost:9876 &
> tail -f ~/logs/rocketmqlogs/namesrv.log
# shutdown
> sh bin/mqshutdown namesrv
Broker
> cd /data/rocketmq-all-4.5.1-bin-release
# startup
> nohup sh bin/mqbroker -c ./conf/broker.conf -n localhost:9876 &
> tail -f ~/logs/rocketmqlogs/broker.log
# shutdown
> sh bin/mqshutdown broker
监控
config [src/main/resources/application.properties]
rocketmq.config.namesrvAddr=
command
> git clone https://github.com/apache/rocketmq-externals.git
> cd rocketmq-console
> mvn clean package -Dmaven.test.skip=true
> nohup java -jar target/rocketmq-console-ng-1.0.1.jar -Dspring.config.location=/data/rocketmq-console/application.properties &
