本文章主要参考《RocketMQ技术内幕：RocketMQ架构设计与实现原理》一书，主要是将自己学习的过程进行记录和梳理，方便以后翻阅。
1. 前言
NameServer主要是监控broker，维护broker相关的信息，并且为生产者和消费者提供主题的路由信息。NameServer通过RouteInfoManager管理broker的节点信息和topic相关的信息。
2. 路由元信息
#org/apache/rocketmq/namesrv/routeinfo/RouteInfoManager
/**
* topic消息队列路由信息
*/
private final HashMap<String/* topic */, List<QueueData>> topicQueueTable;
/**
* broker地址列表信息
*/
private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
/**
* broker集群信息,存储broker的名称
*/
private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable;
/**
* broker状态信息
*/
private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable;
/**
* filterServer列表
*/
private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
在RouteInfoManager中，通过几个集合容器保存相关的信息，分别是：
- topicQueueTable ： topic消息队列路由信息
- brokerAddrTable ：broker地址列表信息
- clusterAddrTable ：broker集群信息,存储broker的名称
- brokerLiveTable ：broker状态信息
- filterServerTable ：filterServer列表
NameServer主要是通过维护这几个容器来实现对broker的监控以及对生产者和消费者的topic路由。
3. 路由注册
#org.apache.rocketmq.broker.BrokerController#start
//broker启动的时候向namesrv发送broker上线的心跳包，并且默认30秒信息一次检查
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
@Override
public void run() {
try {
BrokerController.this.registerBrokerAll(true, false, brokerConfig.isForceRegister());
} catch (Throwable e) {
log.error("registerBrokerAll Exception", e);
}
}
}, 1000 * 10, Math.max(10000, Math.min(brokerConfig.getRegisterNameServerPeriod(), 60000)), TimeUnit.MILLISECONDS);
首先，当broker启动的时候，会启动一个线程在10秒后向NameServer注册自己，并且每三十秒进行一次注册。
#org.apache.rocketmq.broker.out.BrokerOuterAPI#registerBrokerAll
final List<RegisterBrokerResult> registerBrokerResultList = Lists.newArrayList();
//获取namesrv地址列表
List<String> nameServerAddressList = this.remotingClient.getNameServerAddressList();
if (nameServerAddressList != null && nameServerAddressList.size() > 0) {
//构造请求
final RegisterBrokerRequestHeader requestHeader = new RegisterBrokerRequestHeader();
requestHeader.setBrokerAddr(brokerAddr);
requestHeader.setBrokerId(brokerId);
requestHeader.setBrokerName(brokerName);
requestHeader.setClusterName(clusterName);
requestHeader.setHaServerAddr(haServerAddr);
requestHeader.setCompressed(compressed);
//构造body
RegisterBrokerBody requestBody = new RegisterBrokerBody();
requestBody.setTopicConfigSerializeWrapper(topicConfigWrapper);
requestBody.setFilterServerList(filterServerList);
final byte[] body = requestBody.encode(compressed);
final int bodyCrc32 = UtilAll.crc32(body);
requestHeader.setBodyCrc32(bodyCrc32);
final CountDownLatch countDownLatch = new CountDownLatch(nameServerAddressList.size());
//启动多个线程去和所有的namesrv进行通信
for (final String namesrvAddr : nameServerAddressList) {
brokerOuterExecutor.execute(new Runnable() {
@Override
public void run() {
try {
RegisterBrokerResult result = registerBroker(namesrvAddr,oneway, timeoutMills,requestHeader,body);
if (result != null) {
registerBrokerResultList.add(result);
}
log.info("register broker[{}]to name server {} OK", brokerId, namesrvAddr);
} catch (Exception e) {
log.warn("registerBroker Exception, {}", namesrvAddr, e);
} finally {
countDownLatch.countDown();
}
}
});
}
try {
countDownLatch.await(timeoutMills, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
}
}
return registerBrokerResultList;
broker注册的时候最终是调用BrokerOuterAPI#registerBrokerAll()方法进行注册，采用多线程和所有的NameServer进行通信，然后将NameServer返回的信息更新本地信息。更新的信息主要是顺序消息配置、主节点信息等。broker更新本地信息的代码如下：
#org.apache.rocketmq.broker.BrokerController#doRegisterBrokerAll
if (registerBrokerResultList.size() > 0) {
RegisterBrokerResult registerBrokerResult = registerBrokerResultList.get(0);
if (registerBrokerResult != null) {
if (this.updateMasterHAServerAddrPeriodically && registerBrokerResult.getHaServerAddr() != null) {//主从地址更新
this.messageStore.updateHaMasterAddress(registerBrokerResult.getHaServerAddr());
}
this.slaveSynchronize.setMasterAddr(registerBrokerResult.getMasterAddr());
if (checkOrderConfig) {//顺序消息配置更新
this.getTopicConfigManager().updateOrderTopicConfig(registerBrokerResult.getKvTable());
}
}
}
当Broker发送注册请求后，NameServer的请求处理器会接收该请求并做相应的处理，主要是更新RouteInfoManager中管理的几个容器信息。broker注册请求的requestCode为REGISTER_BROKER=103，可以通过该code值找到对应的请求处理逻辑。
#org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#registerBroker
RegisterBrokerResult result = new RegisterBrokerResult();
try {
try {
this.lock.writeLock().lockInterruptibly();
//根据集群名称查找所有的broker
Set<String> brokerNames = this.clusterAddrTable.get(clusterName);
if (null == brokerNames) {
brokerNames = new HashSet<String>();
this.clusterAddrTable.put(clusterName, brokerNames);
}
//将本次注册的brokername加到里面
brokerNames.add(brokerName);
boolean registerFirst = false;
//根据brokerName查找broker的信息，如果不存在，则新建一个
BrokerData brokerData = this.brokerAddrTable.get(brokerName);
if (null == brokerData) {
registerFirst = true;
brokerData = new BrokerData(clusterName, brokerName, new HashMap<Long, String>());
this.brokerAddrTable.put(brokerName, brokerData);
}
Map<Long, String> brokerAddrsMap = brokerData.getBrokerAddrs();
//Switch slave to master: first remove <1, IP:PORT> in namesrv, then add <0, IP:PORT>
//The same IP:PORT must only have one record in brokerAddrTable
//如果broker地址匹配，但是brokerID不匹配，从brokeraddr移除
Iterator<Entry<Long, String>> it = brokerAddrsMap.entrySet().iterator();
while (it.hasNext()) {
Entry<Long, String> item = it.next();
if (null != brokerAddr && brokerAddr.equals(item.getValue()) && brokerId != item.getKey()) {
it.remove();
}
}
//将新的broker地址加入
String oldAddr = brokerData.getBrokerAddrs().put(brokerId, brokerAddr);
registerFirst = registerFirst || (null == oldAddr);
if (null != topicConfigWrapper
&& MixAll.MASTER_ID == brokerId) {//如果是broker中的master节点
if (this.isBrokerTopicConfigChanged(brokerAddr, topicConfigWrapper.getDataVersion())
|| registerFirst) {//如果是第一次，或者broker的topic信息改变了，更新消息队列信息
ConcurrentMap<String, TopicConfig> tcTable =
topicConfigWrapper.getTopicConfigTable();
if (tcTable != null) {
for (Map.Entry<String, TopicConfig> entry : tcTable.entrySet()) {
this.createAndUpdateQueueData(brokerName, entry.getValue());
}
}
}
}
//brokerLiveTable中保存新的broker信息
BrokerLiveInfo prevBrokerLiveInfo = this.brokerLiveTable.put(brokerAddr,
new BrokerLiveInfo(
System.currentTimeMillis(),
topicConfigWrapper.getDataVersion(),
channel,
haServerAddr));
if (null == prevBrokerLiveInfo) {
log.info("new broker registered, {} HAServer: {}", brokerAddr, haServerAddr);
}
if (filterServerList != null) {
if (filterServerList.isEmpty()) {
this.filterServerTable.remove(brokerAddr);
} else {
this.filterServerTable.put(brokerAddr, filterServerList);
}
}
if (MixAll.MASTER_ID != brokerId) {
String masterAddr = brokerData.getBrokerAddrs().get(MixAll.MASTER_ID);
if (masterAddr != null) {
BrokerLiveInfo brokerLiveInfo = this.brokerLiveTable.get(masterAddr);
if (brokerLiveInfo != null) {
result.setHaServerAddr(brokerLiveInfo.getHaServerAddr());
result.setMasterAddr(masterAddr);
}
}
}
} finally {
this.lock.writeLock().unlock();
}
} catch (Exception e) {
log.error("registerBroker Exception", e);
}
return result;
上面这段代码主要是NameServer在接收到Broker的注册请求后的处理逻辑，其主要任务就是将clusterAddrTable、brokerAddrTable、topicQueueTable、brokerLiveTable、filterServerTable更新。到此，broker注册流程完成。
4. 路由删除
当Broker失效后，NameServer需要将其从几个维护Broker信息的容器中删除，NameServer有两个地方会触发删除，一个是NameServer会定时扫描brokerLiveTable检查上一次的更新时间和当前系统时间如果超过120秒，则移除该broker的信息，另一个就是当broker正常被关闭的时候，会发出unregisterBroker的指令，NameServer会移除该broker的信息。
4.1 NameServer定时扫描移除Broker
#org.apache.rocketmq.namesrv.NamesrvController#initialize
/**
* 创建一个定时任务，每隔10秒扫描一次broker，移除处于不激活状态的broker
*/
this.scheduledExecutorService.scheduleAtFixedRate(new Runnable(){
@Override
public void run() {
NamesrvController.this.routeInfoManager.scanNotActiveBroker();
}
}, 5, 10, TimeUnit.SECONDS);
#org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#scanNotActiveBroker
Iterator<Entry<String, BrokerLiveInfo>> it = this.brokerLiveTable.entrySet().iterator();
while (it.hasNext()) {
Entry<String, BrokerLiveInfo> next = it.next();
long last = next.getValue().getLastUpdateTimestamp();
if ((last + BROKER_CHANNEL_EXPIRED_TIME) < System.currentTimeMillis()) {//如果broker最后一次更新的时间与当前时间间隔超过120秒，则移除
RemotingUtil.closeChannel(next.getValue().getChannel());
it.remove();
log.warn("The broker channel expired, {} {}ms", next.getKey(), BROKER_CHANNEL_EXPIRED_TIME);
this.onChannelDestroy(next.getKey(), next.getValue().getChannel());
}
}
在NameServer启动的时候，会创建一个定时任务，每隔10秒钟去扫描一次brokerLiveTable,如果broker的更新时间和当前时间相差120秒，则将broker从NameServer中移除。具体的移除逻辑在onChannelDestroy()方法中，主要是将clusterAddrTable、brokerAddrTable、topicQueueTable、brokerLiveTable、filterServerTable中对应的Broker信息移除掉。
4.2. Broker正常关闭取消注册
当Broker正常关闭的时候，会向NameServer发送取消注册的流程，NameServer接收到该请求后，会将对应的Broker移除。
#org.apache.rocketmq.broker.out.BrokerOuterAPI#unregisterBroker
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.UNREGISTER_BROKER, requestHeader);
RemotingCommand response = this.remotingClient.invokeSync(namesrvAddr, request, 3000);
在关闭broker的时候，取消注册最主要的就是发送了一个requestCode为UNREGISTER_BROKER 104 的请求。然后NameServer会处理该请求。
NameServer最终处理该请求的逻辑是在RouteInfoManager#unregisterBroker()方法中，主要也是将clusterAddrTable、brokerAddrTable、topicQueueTable、brokerLiveTable、filterServerTable中对应的Broker信息移除掉。
5. 路由发现
路由发现是指当生产者或者消费者发送或者消费消息的时候，需要知道topic相关的信息，客户端会发送一个GET_ROUTEINTO_BY_TOPIC的请求，请求NameServer，更新自己本地的topic缓存信息。
6. 参考资料
- RocketMQ技术内幕：RocketMQ架构设计与实现原理
