rocketMQ
rocketMq安装
rocketMq无法启动namesrv，namebroker
内存问题，runbroker，runserver的配置文件里java内存xms，xmx设置的过大
rocketMQ启动
-
mqbroker启动以及查看日志
nohup ./mqbroker -n localhost:9876 & 启动mqbroker官方是通过内网ip进行连接，会导致客户端无法远程连接，在修改broker.conf文件后，使用如下命令 nohup sh bin/mqbroker -n 175.24.53.251:9876 autoCreateTopicEnable=true -c /var/www/.../.../conf/broker.conf & tail -f ~/logs/rocketmqlogs/broker.log -
mqnamesrv启动以及查看日志
nohup ./mqnamesrv & tail -f ~/logs/rocketmqlogs/namesrv.log
项目中间疑难点
-
消息被重复消费？
查看消息体里的内容，查看其offset
-
事务消息队列
MQProducer源码解析
-
启动start()
-
@Override public void start() throws MQClientException { this.defaultMQProducerImpl.start(); }这里初始化了MQClientFactory
getAndCreateMQClientInstance方法首先取出之前生成的ClientId,ClientID是由客户端ip+@+实例名构建的，然后去factoryTable去尝试获取，如果为空则生成一个
对生产者的生产者组组名进行检查并注册，检查已存在的话就报错
注册成功后将生产者的topic与发布关系放入topicPublishInfoTable表中然后启动MQ客户端
总体流程如下
-
-
发送消息send()
-
整体发送消息的逻辑是由defaultMQProducerImpl来完成的，也就是之前mq客户端启动时的启动的生产者线程
msg:消息体 CommunicationMode:发送类别，有同步方式SYNC，异步方式ASYNC，单向模式ONR WAY sendCallbcak:回调函数，一般用于异步发送消息方式 timeout：超时时间 -
第一步：检查消息体msg的合法性
主要是检查消息以及消息内容和topic是否为空，长度是否超出最大限制以及名称是否合法等问题
-
第二步：通过topic名称获取broker路由信息
private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) { //1.首先会去缓存中获取该topic路由信息 TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic); //2.1如果为空，则去访问NameServer去获取topic路由信息 if (null == topicPublishInfo || !topicPublishInfo.ok()) { this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo()); this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic); topicPublishInfo = this.topicPublishInfoTable.get(topic); } //2.2如果不为空，则检查topic路由信息是否过期，没过期直接返回，过期了去NameServer查询 if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) { return topicPublishInfo; } else { this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer); topicPublishInfo = this.topicPublishInfoTable.get(topic); return topicPublishInfo; } } -
第三步：从返回的broker路由信息中选取一个消息队列来达到负载均衡
在这里首先它会进行一个容错策略判断，默认容错策略是false，也就是不进入容错策略语句分支中，如果容错策略为true，那么选择队列的策略是会去先选择lastBrokerName的消息队列topicPublishInfo：topic路由信息 lastBrokerName：上一个broker名称 public MessageQueue selectOneMessageQueue(final String lastBrokerName) { //如果上一个brokername为空直接随便选取一个消息队列，如果不为空为了负载均衡我们应尽可能避免这一次继续使用上一个brokername if (lastBrokerName == null) { return selectOneMessageQueue(); } else { //通过sendWhichQueue.getAndIncrement()方法获取自增字段，这样每次获取的index都会不同，达到了负载均衡的效果 int index = this.sendWhichQueue.getAndIncrement(); for (int i = 0; i < this.messageQueueList.size(); i++) { int pos = Math.abs(index++) % this.messageQueueList.size(); if (pos < 0) pos = 0; MessageQueue mq = this.messageQueueList.get(pos); //如果不等于上一次的brokername则直接返回该消息队列 if (!mq.getBrokerName().equals(lastBrokerName)) { return mq; } } //当我们只设置了一个broker那么只能使用该broker的消息队列，我们的topic是分布在多个broker中的，而每个broker中有多个topic，每个topic有多个消息队列，当我们只配置了一个broker，那么topic只能存在于这一个broker中 return selectOneMessageQueue(); } } -
第四步：发送消息
-
首先通过brokername获取broker
-
然后对msg结构体进行处理，对msg分配唯一ID与他的properties中
-
判断是否设置有钩子函数，如果有则设置发送消息上下文
-
封装请求头
-
对消息进行发送，这里选取ASYNC模式进行分析
-
-
private void sendMessageAsync(
final String addr,
final String brokerName,
final Message msg,
final long timeoutMillis,
final RemotingCommand request,
final SendCallback sendCallback,
final TopicPublishInfo topicPublishInfo,
final MQClientInstance instance,
final int retryTimesWhenSendFailed,
final AtomicInteger times,
final SendMessageContext context,
final DefaultMQProducerImpl producer
) throws InterruptedException, RemotingException {
this.remotingClient.invokeAsync(addr, request, timeoutMillis, new InvokeCallback() {
@Override
public void operationComplete(ResponseFuture responseFuture) {
RemotingCommand response = responseFuture.getResponseCommand();
//如果没有设置回调函数但response不为空，则通过processSendResponse来处理response获得sendResult
if (null == sendCallback && response != null) {
try {
SendResult sendResult = MQClientAPIImpl.this.processSendResponse(brokerName, msg, response);
//如果上下文不为空，则执行钩子函数的消息发送后after方法
if (context != null && sendResult != null) {
context.setSendResult(sendResult);
context.getProducer().executeSendMessageHookAfter(context);
}
} catch (Throwable e) {
}
//更新故障的broker
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), false);
return;
}
//如果设置了回调函数且response不为空
if (response != null) {
try {
//获取sendResult
SendResult sendResult = MQClientAPIImpl.this.processSendResponse(brokerName, msg, response);
assert sendResult != null;
if (context != null) {
context.setSendResult(sendResult);
//执行发送后的钩子函数
context.getProducer().executeSendMessageHookAfter(context);
}
try {
//通过回调函数对sendResult进行处理
sendCallback.onSuccess(sendResult);
} catch (Throwable e) {
}
//更新错误broker
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), false);
} catch (Exception e) {
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), true);
onExceptionImpl(brokerName, msg, 0L, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, e, context, false, producer);
}
} else {
//resonse为空，判断是超时还是发送失败还是不明原因
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), true);
if (!responseFuture.isSendRequestOK()) {
MQClientException ex = new MQClientException("send request failed", responseFuture.getCause());
onExceptionImpl(brokerName, msg, 0L, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
} else if (responseFuture.isTimeout()) {
MQClientException ex = new MQClientException("wait response timeout " + responseFuture.getTimeoutMillis() + "ms",
responseFuture.getCause());
onExceptionImpl(brokerName, msg, 0L, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
} else {
MQClientException ex = new MQClientException("unknow reseaon", responseFuture.getCause());
onExceptionImpl(brokerName, msg, 0L, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
}
}
}
});
}
mvn install:install-file -Dfile=d:\setup\rocketmq-common-4.5.0.jar -DgroupId=org.apache.rocketmq -DartifactId=rocketmq-common -Dversion=4.5.0 -Dpackaging=jar
（本文仅用于记录平时个人学习心得，如有误导，恳请指教）
