- 消息接收：消息接收是指接收
producer的消息，处理类是SendMessageProcessor，将消息写入到commigLog文件后，接收流程处理完毕； - 消息分发：
broker处理消息分发的类是ReputMessageService，它会启动一个线程，不断地将commitLong分到到对应的consumerQueue，这一步操作会写两个文件：consumerQueue与indexFile，写入后，消息分发流程处理 完毕； - 消息投递：消息投递是指将消息发往
consumer的流程，consumer会发起获取消息的请求，broker收到请求后，调用PullMessageProcessor类处理，从consumerQueue文件获取消息，返回给consumer后，投递流程处理完毕。
以上就是rocketMq处理消息的流程了，接下来我们就从源码来分析消息分发的实现。
一.消息分发线程
在前文# RocketMq源码分析(四):Broker启动流程中,我们得知核心流程有
brokerController.start()方法
public void start() throws Exception {
....
/**
* 启动一些基础服务
* 比如: 启动消息存储/接受producer发消息的netty
* todo messageStore.start() messageStore:从commitLog文件读取数据,存入consumerQueue文件中
*/
startBasicService();
继续进入方法,会看到
if (this.messageStore != null) {
this.messageStore.start();
}
这里就是分发消息的启动入口 继续进入
public void start() throws Exception {
if (!messageStoreConfig.isEnableDLegerCommitLog() && !this.messageStoreConfig.isDuplicationEnable()) {
this.haService.init(this);
}
if (messageStoreConfig.isTransientStorePoolEnable()) {
this.transientStorePool.init();
}
this.allocateMappedFileService.start();
this.indexService.start();
lock = lockFile.getChannel().tryLock(0, 1, false);
if (lock == null || lock.isShared() || !lock.isValid()) {
throw new RuntimeException("Lock failed,MQ already started");
}
lockFile.getChannel().write(ByteBuffer.wrap("lock".getBytes(StandardCharsets.UTF_8)));
lockFile.getChannel().force(true);
if (this.getMessageStoreConfig().isDuplicationEnable()) {
// todo 设置commitLog的偏移量
this.reputMessageService.setReputFromOffset(this.commitLog.getConfirmOffset());
} else {
// It is [recover]'s responsibility to fully dispatch the commit log data before the max offset of commit log.
this.reputMessageService.setReputFromOffset(this.commitLog.getMaxOffset());
}
//todo 消息分发操作，启动新线程来处理
this.reputMessageService.start();
会触发reputMessageService.start()方法,来开启线程,处理
会调用ReputMessageService类的run方法,如下:
public void run() {
DefaultMessageStore.LOGGER.info(this.getServiceName() + " service started");
while (!this.isStopped()) {
try {
Thread.sleep(1);
// 开启线程处理commitLog 文件中的数据
this.doReput();
} catch (Exception e) {
DefaultMessageStore.LOGGER.warn(this.getServiceName() + " service has exception. ", e);
}
}
DefaultMessageStore.LOGGER.info(this.getServiceName() + " service end");
}
继续进入doReput()方法
private void doReput() {
// 处理 reputFromOffset
if (this.reputFromOffset < DefaultMessageStore.this.commitLog.getMinOffset()) {
LOGGER.warn("The reputFromOffset={} is smaller than minPyOffset={}, this usually indicate that the dispatch behind too much and the commitlog has expired.",
this.reputFromOffset, DefaultMessageStore.this.commitLog.getMinOffset());
this.reputFromOffset = DefaultMessageStore.this.commitLog.getMinOffset();
}
for (boolean doNext = true; this.isCommitLogAvailable() && doNext; ) {
// todo 从CommitLog中获取需要进行转发的消息
SelectMappedBufferResult result = DefaultMessageStore.this.commitLog.getData(reputFromOffset);
if (result != null) {
try {
this.reputFromOffset = result.getStartOffset();
for (int readSize = 0; readSize < result.getSize() && reputFromOffset < DefaultMessageStore.this.getConfirmOffset() && doNext; ) {
// 检验数据
DispatchRequest dispatchRequest =
DefaultMessageStore.this.commitLog.checkMessageAndReturnSize(result.getByteBuffer(), false, false, false);
int size = dispatchRequest.getBufferSize() == -1 ? dispatchRequest.getMsgSize() : dispatchRequest.getBufferSize();
if (reputFromOffset + size > DefaultMessageStore.this.getConfirmOffset()) {
doNext = false;
break;
}
if (dispatchRequest.isSuccess()) {
if (size > 0) {
/**
* todo 分发消息
* 就是把消息的相关写入ConsumeQueue与IndexFile两个文件中
*/
DefaultMessageStore.this.doDispatch(dispatchRequest);
// 长轮询：如果有消息到了主节点，并且开启了长轮询
if (DefaultMessageStore.this.brokerConfig.isLongPollingEnable()
&& DefaultMessageStore.this.messageArrivingListener != null) {
// 调用NotifyMessageArrivingListener的arriving方法
DefaultMessageStore.this.messageArrivingListener.arriving(dispatchRequest.getTopic(),
....
}
.
} else if (!dispatchRequest.isSuccess()) {
...
}
}
}
} finally {
result.release();
}
} else {
doNext = false;
}
}
}
继续进入方法doDispatch(dispatchRequest)
public void doDispatch(DispatchRequest req) {
/**
* 在DefaultMessageStore的构造方法中,
* this.dispatcherList.addLast(new CommitLogDispatcherBuildConsumeQueue());
* this.dispatcherList.addLast(new CommitLogDispatcherBuildIndex());
*/
for (CommitLogDispatcher dispatcher : this.dispatcherList) {
dispatcher.dispatch(req);
}
}
二.消息写入consumerQueue文件和indexFile文件
上一章的最后,我们进入到dispatcherList的dispatch方法,list中有两个类,分别是
- CommitLogDispatcherBuildConsumeQueue
- CommitLogDispatcherBuildIndex
我们分别看代码
2.1 CommitLogDispatcherBuildConsumeQueue
class CommitLogDispatcherBuildConsumeQueue implements CommitLogDispatcher {
@Override
public void dispatch(DispatchRequest request) {
final int tranType = MessageSysFlag.getTransactionValue(request.getSysFlag());
switch (tranType) {
case MessageSysFlag.TRANSACTION_NOT_TYPE:
case MessageSysFlag.TRANSACTION_COMMIT_TYPE:
// 将消息在commitLog文件的位置、tags等信息写入ConsumerQueue文件
DefaultMessageStore.this.putMessagePositionInfo(request);
break;
case MessageSysFlag.TRANSACTION_PREPARED_TYPE:
case MessageSysFlag.TRANSACTION_ROLLBACK_TYPE:
break;
}
}
}
2.2 CommitLogDispatcherBuildIndex
class CommitLogDispatcherBuildIndex implements CommitLogDispatcher {
@Override
public void dispatch(DispatchRequest request) {
if (DefaultMessageStore.this.messageStoreConfig.isMessageIndexEnable()) {
DefaultMessageStore.this.indexService.buildIndex(request);
}
}
}
需要注意的是，在这两个文件中，写入的仅是消息的位置信息，完整的消息内容仅在commitLog中保存。
这两个写入文件的代码逻辑都比较多,会计算偏移量/数据大小等信息,有兴趣的小伙伴可以去深入看看
三,总结
本章结合第六章,主要是讲解broker收到消息后,会把消息先写到
commitLog文件中,然后有其他定时任务ReputMessageService,根据commitLog中的偏移量,把新提交的消息,写入到consumerQuene文件和indexFile文件中,其中consumerQuene是提供consumer进行消费的
就到这吧,要看世界杯去了,呜呼~
