准备工作
在dubbo官网下载源码，3.2.6版本。启动Zookeeper和服务提供者provider。
consumer进行服务发现调试入口：org.apache.dubbo.demo.consumer.Application#runWithBootstrap
private static void runWithBootstrap() {
// consumer的引用配置
ReferenceConfig<DemoService> reference = new ReferenceConfig<>();
// 设置要引用的服务接口（当一系列接口被producer打包出口后，会通过网络传输，将信息存在zk中，consumer实际消费时，会将zk中的接口信息取出，并通过代理构造此处的ReferenceConfig）
reference.setInterface(DemoService.class);
reference.setGeneric("true");
// 引导启动组件，3.0.x还没有该组件
DubboBootstrap bootstrap = DubboBootstrap.getInstance();
bootstrap
.application(new ApplicationConfig("dubbo-demo-api-consumer"))
.registry(new RegistryConfig(REGISTRY_URL))
.protocol(new ProtocolConfig(CommonConstants.DUBBO, -1))
.reference(reference)
// 从这里进入调试
.start();
DemoService demoService = bootstrap.getCache().get(reference);
String message = demoService.sayHello("dubbo");
System.out.println(message);
// generic invoke
GenericService genericService = (GenericService) demoService;
Object genericInvokeResult = genericService.$invoke(
"sayHello", new String[] {String.class.getName()}, new Object[] {"dubbo generic invoke"});
System.out.println(genericInvokeResult.toString());
}
从这个方法进来，做了一些准备工作，和服务发布与注册开始时几乎相同，在此不在一一赘述。
开始调试
在startSync()方法中，走referServices()，再向下一直到org.apache.dubbo.config.ReferenceConfig#init(boolean)
protected synchronized void init(boolean check) {
.
.
.
.
// 准备工作，填充配置
// 核心流程
ref = createProxy(referenceParameters);
serviceMetadata.setTarget(ref);
serviceMetadata.addAttribute(PROXY_CLASS_REF, ref);
consumerModel.setDestroyRunner(getDestroyRunner());
consumerModel.setProxyObject(ref);
consumerModel.initMethodModels();
if (check) {
checkInvokerAvailable(0);
}
} catch (Throwable t) {
logAndCleanup(t);
throw t;
}
initialized = true;
}
可以看到，这里要创建接口实现类的代理，赋值给ref（接口代理的引用）
在createProxy方法中，我们直接来看createInvoker()方法
org.apache.dubbo.config.ReferenceConfig#createInvoker
/**
* \create a reference invoker
*/
@SuppressWarnings({"unchecked", "rawtypes"})
private void createInvoker() {
if (urls.size() == 1) {
URL curUrl = urls.get(0);
invoker = protocolSPI.refer(interfaceClass, curUrl);
// registry url, mesh-enable and unloadClusterRelated is true, not need Cluster.
if (!UrlUtils.isRegistry(curUrl) && !curUrl.getParameter(UNLOAD_CLUSTER_RELATED, false)) {
List<Invoker<?>> invokers = new ArrayList<>();
invokers.add(invoker);
invoker = Cluster.getCluster(getScopeModel(), Cluster.DEFAULT)
.join(new StaticDirectory(curUrl, invokers), true);
}
} else {
List<Invoker<?>> invokers = new ArrayList<>();
URL registryUrl = null;
for (URL url : urls) {
// For multi-registry scenarios, it is not checked whether each referInvoker is available.
// Because this invoker may become available later.
invokers.add(protocolSPI.refer(interfaceClass, url));
if (UrlUtils.isRegistry(url)) {
// use last registry url
registryUrl = url;
}
}
if (registryUrl != null) {
// registry url is available
// for multi-subscription scenario, use 'zone-aware' policy by default
String cluster = registryUrl.getParameter(CLUSTER_KEY, ZoneAwareCluster.NAME);
// The invoker wrap sequence would be: ZoneAwareClusterInvoker(StaticDirectory) ->
// FailoverClusterInvoker
// (RegistryDirectory, routing happens here) -> Invoker
invoker = Cluster.getCluster(registryUrl.getScopeModel(), cluster, false)
.join(new StaticDirectory(registryUrl, invokers), false);
} else {
// not a registry url, must be direct invoke.
if (CollectionUtils.isEmpty(invokers)) {
throw new IllegalArgumentException("invokers == null");
}
URL curUrl = invokers.get(0).getUrl();
String cluster = curUrl.getParameter(CLUSTER_KEY, Cluster.DEFAULT);
invoker =
Cluster.getCluster(getScopeModel(), cluster).join(new StaticDirectory(curUrl, invokers), true);
}
}
}
这里invoker = protocolSPI.refer(interfaceClass, curUrl)有一个SPI机制，调试不好直接跳到对应实现，直接说明此处跳到RegistryProtocol中，可以在RegistryProtocol的refer方法打断点。
@Override
@SuppressWarnings("unchecked")
public <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException {
url = getRegistryUrl(url);
Registry registry = getRegistry(url);
if (RegistryService.class.equals(type)) {
return proxyFactory.getInvoker((T) registry, type, url);
}
// group="a,b" or group="*"
Map<String, String> qs = (Map<String, String>) url.getAttribute(REFER_KEY);
String group = qs.get(GROUP_KEY);
if (StringUtils.isNotEmpty(group)) {
if ((COMMA_SPLIT_PATTERN.split(group)).length > 1 || "*".equals(group)) {
return doRefer(Cluster.getCluster(url.getScopeModel(), MergeableCluster.NAME), registry, type, url, qs);
}
}
Cluster cluster = Cluster.getCluster(url.getScopeModel(), qs.get(CLUSTER_KEY));
return doRefer(cluster, registry, type, url, qs);
}
此方法中，Registry registry = getRegistry(url);获取到ZookeeperRegistry，向下都到doRefer
protected <T> Invoker<T> doRefer(
Cluster cluster, Registry registry, Class<T> type, URL url, Map<String, String> parameters) {
Map<String, Object> consumerAttribute = new HashMap<>(url.getAttributes());
consumerAttribute.remove(REFER_KEY);
String p = isEmpty(parameters.get(PROTOCOL_KEY)) ? CONSUMER : parameters.get(PROTOCOL_KEY);
URL consumerUrl = new ServiceConfigURL(
p,
null,
null,
parameters.get(REGISTER_IP_KEY),
0,
getPath(parameters, type),
parameters,
consumerAttribute);
url = url.putAttribute(CONSUMER_URL_KEY, consumerUrl);
ClusterInvoker<T> migrationInvoker = getMigrationInvoker(this, cluster, registry, type, url, consumerUrl);
return interceptInvoker(migrationInvoker, url, consumerUrl);
}
创建consumerUrl
consumer://10.1.xx.219/org.apache.dubbo.demo.DemoService?application=dubbo-demo-api-consumer&background=false&dubbo=2.0.2&executor-management-mode=isolation&file-cache=true&generic=true&interface=org.apache.dubbo.demo.DemoService&pid=3700®ister.ip=10.1.68.219&release=&side=consumer&sticky=false×tamp=1742004447403&unloadClusterRelated=false
创建MigrationInvoker用于负责处理服务迁移（负责处理服务级Invoker和接口级Invoker的迁移），其中包含了以下三个参数。
private volatile ClusterInvoker invoker; // 用来记录接口级ClusterInvoker private volatile ClusterInvoker serviceDiscoveryInvoker; // 用来记录应用级的ClusterInvoker private volatile ClusterInvoker currentAvailableInvoker; // 用来记录当前使用的ClusterInvoker，要么是接口级，要么应用级
之后，执行interceptInvoker()来利用监听器，更改Invoker的行为
protected <T> Invoker<T> interceptInvoker(ClusterInvoker<T> invoker, URL url, URL consumerUrl) {
List<RegistryProtocolListener> listeners = findRegistryProtocolListeners(url);
if (CollectionUtils.isEmpty(listeners)) {
return invoker;
}
for (RegistryProtocolListener listener : listeners) {
listener.onRefer(this, invoker, consumerUrl, url);
}
return invoker;
}
通过url获取监听器RegistryProtocolListener，此处的listener是MigrationRuleListener
MigrationRuleListener ：Listens to {@MigrationRule} from Config Center
之后，执行onRefer()，相当于一个回调，在订阅服务时，去通知Listener
@Override
public void onRefer(
RegistryProtocol registryProtocol, ClusterInvoker<?> invoker, URL consumerUrl, URL registryURL) {
MigrationRuleHandler<?> migrationRuleHandler =
ConcurrentHashMapUtils.computeIfAbsent(handlers, (MigrationInvoker<?>) invoker, _key -> {
((MigrationInvoker<?>) invoker).setMigrationRuleListener(this);
return new MigrationRuleHandler<>((MigrationInvoker<?>) invoker, consumerUrl);
});
migrationRuleHandler.doMigrate(rule);
}
Invoker迁移机制
在Dubbo 3的迁移机制中，为了支持从接口级平滑过渡到应用级服务发现，需要同时维护两种服务发现的Invoker。refreshInterfaceInvoker处理旧模式，refreshServiceDiscoveryInvoker处理新模式，然后根据迁移规则动态选择优先使用的Invoker，确保兼容性和平滑迁移
此方法构建一个migrationRuleHandler去依据迁移rule处理迁移，此时的rule内部还都是null，step是应用级优先（APPLICATION_FIRST）
public synchronized void doMigrate(MigrationRule rule) {
if (migrationInvoker instanceof ServiceDiscoveryMigrationInvoker) {
refreshInvoker(MigrationStep.FORCE_APPLICATION, 1.0f, rule);
return;
}
// initial step : APPLICATION_FIRST
MigrationStep step = MigrationStep.APPLICATION_FIRST;
float threshold = -1f;
try {
step = rule.getStep(consumerURL);
threshold = rule.getThreshold(consumerURL);
} catch (Exception e) {
logger.error(
REGISTRY_NO_PARAMETERS_URL, "", "", "Failed to get step and threshold info from rule: " + rule, e);
}
// 去根据rule刷新Invoker
if (refreshInvoker(step, threshold, rule)) {
// refresh success, update rule
setMigrationRule(rule);
}
}
在这里回去刷新Invoker，执行refreshInvoker()
在refreshInvoker()中，按照step，去执行下面这个方法
@Override
public void migrateToApplicationFirstInvoker(MigrationRule newRule) {
CountDownLatch latch = new CountDownLatch(0);
// 刷新接口级Invoker
refreshInterfaceInvoker(latch);
// 刷新应用级Invoker
refreshServiceDiscoveryInvoker(latch);
// directly calculate preferred invoker, will not wait until address notify
// calculation will re-occurred when address notify later
calcPreferredInvoker(newRule);
}
处理接口级Invoker
refreshInterfaceInvoker()
protected void refreshInterfaceInvoker(CountDownLatch latch) {
clearListener(invoker);
if (needRefresh(invoker)) {
if (logger.isDebugEnabled()) {
logger.debug("Re-subscribing interface addresses for interface " + type.getName());
}
if (invoker != null) {
invoker.destroy();
}
// 从这里进去
invoker = registryProtocol.getInvoker(cluster, registry, type, url);
}
setListener(invoker, () -> {
latch.countDown();
if (reportService.hasReporter()) {
reportService.reportConsumptionStatus(reportService.createConsumptionReport(
consumerUrl.getServiceInterface(),
consumerUrl.getVersion(),
consumerUrl.getGroup(),
"interface"));
}
if (step == APPLICATION_FIRST) {
calcPreferredInvoker(rule);
}
});
}
直接看invoker = registryProtocol.getInvoker(cluster, registry, type, url)
public <T> ClusterInvoker<T> getInvoker(Cluster cluster, Registry registry, Class<T> type, URL url) {
DynamicDirectory<T> directory = new RegistryDirectory<>(type, url);
return doCreateInvoker(directory, cluster, registry, type);
}
创建了一个DynamicDirectory动态注册表，之后执行doCreateInvoker
protected <T> ClusterInvoker<T> doCreateInvoker(
DynamicDirectory<T> directory, Cluster cluster, Registry registry, Class<T> type) {
directory.setRegistry(registry);
directory.setProtocol(protocol);
// all attributes of REFER_KEY
Map<String, String> parameters =
new HashMap<>(directory.getConsumerUrl().getParameters());
URL urlToRegistry = new ServiceConfigURL(
parameters.get(PROTOCOL_KEY) == null ? CONSUMER : parameters.get(PROTOCOL_KEY),
parameters.remove(REGISTER_IP_KEY),
0,
getPath(parameters, type),
parameters);
urlToRegistry = urlToRegistry.setScopeModel(directory.getConsumerUrl().getScopeModel());
urlToRegistry = urlToRegistry.setServiceModel(directory.getConsumerUrl().getServiceModel());
if (directory.isShouldRegister()) {
directory.setRegisteredConsumerUrl(urlToRegistry);
// 注册consumer节点
registry.register(directory.getRegisteredConsumerUrl());
}
directory.buildRouterChain(urlToRegistry);
// 订阅zk节点
directory.subscribe(toSubscribeUrl(urlToRegistry));
return (ClusterInvoker<T>) cluster.join(directory, true);
}
向zk注册consumer节点
此时，会先去注册consumer节点，执行registry.register(directory.getRegisteredConsumerUrl())
@Override
public void register(URL url) {
try {
if (registry != null) {
registry.register(url);
}
} finally {
if (!UrlUtils.isConsumer(url)) {
listenerEvent(serviceListener -> serviceListener.onRegister(url, registry));
}
}
}
向下调用，跳转到org.apache.dubbo.registry.zookeeper.ZookeeperRegistry#doRegister
@Override
public void doRegister(URL url) {
try {
checkDestroyed();
zkClient.create(toUrlPath(url), url.getParameter(DYNAMIC_KEY, true), true);
} catch (Throwable e) {
throw new RpcException(
"Failed to register " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
根据url去创建zk节点， path如下
/dubbo/org.apache.dubbo.demo.DemoService/consumers/consumer%3A%2F%2F10.1.68.219%2Forg.apache.dubbo.demo.DemoService%3Fapplication%3Ddubbo-demo-api-consumer%26background%3Dfalse%26category%3Dconsumers%26check%3Dfalse%26dubbo%3D2.0.2%26executor-management-mode%3Disolation%26file-cache%3Dtrue%26generic%3Dtrue%26interface%3Dorg.apache.dubbo.demo.DemoService%26pid%3D3700%26release%3D%26side%3Dconsumer%26sticky%3Dfalse%26timestamp%3D1742004447403%26unloadClusterRelated%3Dfalse
具体创建节点的流程可以看org.apache.dubbo.remoting.zookeeper.AbstractZookeeperClient#create，此流程在服务发布与注册中提到过，在此不做赘述。
之后，返回去看directory.subscribe(toSubscribeUrl(urlToRegistry));
provider服务订阅与发现（接口级服务发现）
subscribe()方法中，有下面一段代码
MetricsEventBus.post(RegistryEvent.toSubscribeEvent(applicationModel, registryClusterName), () -> {
super.subscribe(url);
return null;
});
这部分利用MetricsEventBus去向所有注册订阅者发布RegistryEvent事件，并返回处理结果，也就是super.subscribe(url)的结果
继续向下看subscribe流程到org.apache.dubbo.registry.zookeeper.ZookeeperRegistry#doSubscribe
@Override
public void doSubscribe(final URL url, final NotifyListener listener) {
try {
// 此处省略
} else {
CountDownLatch latch = new CountDownLatch(1);
try {
List<URL> urls = new ArrayList<>();
/*
在默认设置中，url是一个consumer时，category可以是以下三个
/dubbo/[service name]/providers,
/dubbo/[service name]/configurators
/dubbo/[service name]/routers
*/
for (String path : toCategoriesPath(url)) {
ConcurrentMap<NotifyListener, ChildListener> listeners = ConcurrentHashMapUtils.computeIfAbsent(
zkListeners, url, k -> new ConcurrentHashMap<>());
ChildListener zkListener = ConcurrentHashMapUtils.computeIfAbsent(
listeners, listener, k -> new RegistryChildListenerImpl(url, k, latch));
if (zkListener instanceof RegistryChildListenerImpl) {
((RegistryChildListenerImpl) zkListener).setLatch(latch);
}
// create "directories".
zkClient.create(path, false, true);
// Add children (i.e. service items).
List<String> children = zkClient.addChildListener(path, zkListener);
if (children != null) {
// The invocation point that may cause 1-1.
urls.addAll(toUrlsWithEmpty(url, path, children));
}
}
notify(url, listener, urls);
} finally {
// tells the listener to run only after the sync notification of main thread finishes.
latch.countDown();
}
}
} catch (Throwable e) {
throw new RpcException(
"Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
上述过程，对不同category的path进行循环
第一个category：providers，走到create时，因zk已存在该节点，所以不会再去zk创建该节点，继续向下，addChildListener，向该节点下添加监听器，在zk是设置Watcher，可参考client.getChildren().usingWatcher(listener).forPath(path)代码，获得目标节点下的子节点，完成向zk节点的订阅。订阅后将 toUrlsWithEmpty(url, path, children)处理后的url（category是providers的url）放入urls
第二个category是configurators
第三个是routers，过程和上述相同，最后都放入到urls。
拿到订阅后的urls，去执行notify()
一路跳转到org.apache.dubbo.registry.support.AbstractRegistry#notify(org.apache.dubbo.common.URL, org.apache.dubbo.registry.NotifyListener, java.util.List<org.apache.dubbo.common.URL>)
/**
* Notify changes from the provider side.
*
* @param url consumer side url
* @param listener listener
* @param urls provider latest urls
*/
protected void notify(URL url, NotifyListener listener, List<URL> urls) {
if (url == null) {
throw new IllegalArgumentException("notify url == null");
}
if (listener == null) {
throw new IllegalArgumentException("notify listener == null");
}
if ((CollectionUtils.isEmpty(urls)) && !ANY_VALUE.equals(url.getServiceInterface())) {
// 1-4 Empty address.
logger.warn(REGISTRY_EMPTY_ADDRESS, "", "", "Ignore empty notify urls for subscribe url " + url);
return;
}
if (logger.isInfoEnabled()) {
logger.info("Notify urls for subscribe url " + url + ", url size: " + urls.size());
}
// keep every provider's category.
Map<String, List<URL>> result = new HashMap<>();
for (URL u : urls) {
if (UrlUtils.isMatch(url, u)) {
String category = u.getCategory(DEFAULT_CATEGORY);
List<URL> categoryList = result.computeIfAbsent(category, k -> new ArrayList<>());
categoryList.add(u);
}
}
if (result.size() == 0) {
return;
}
Map<String, List<URL>> categoryNotified = notified.computeIfAbsent(url, u -> new ConcurrentHashMap<>());
for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
String category = entry.getKey();
List<URL> categoryList = entry.getValue();
categoryNotified.put(category, categoryList);
listener.notify(categoryList);
// We will update our cache file after each notification.
// When our Registry has a subscribed failure due to network jitter, we can return at least the existing
// cache URL.
if (localCacheEnabled) {
saveProperties(url);
}
}
}
本方法会对三种category对应的url进行循环notify，最后再对cache file（服务器本地存文件）进行更新。
向下继续调用org.apache.dubbo.registry.integration.RegistryDirectory#notify
此方法中会处理url，加载扩展机制，但核心的还是调用refreshOverrideAndInvoker(providerURLs)方法
// RefreshOverrideAndInvoker will be executed by registryCenter and configCenter, so it should be synchronized.
@Override
protected synchronized void refreshOverrideAndInvoker(List<URL> urls) {
// mock zookeeper://xxx?mock=return null
this.directoryUrl = overrideWithConfigurator(getOriginalConsumerUrl());
refreshInvoker(urls);
}
当category为configurators和routers时，入参urls都是empty的，到category为providers时，urls有具体数据，此时urls数据示例如下
dubbo://10.1.xx.219:20880/org.apache.dubbo.demo.DemoService?application=dubbo-demo-api-provider&background=false&category=providers,configurators,routers&check=false&deprecated=false&dubbo=2.0.2&dynamic=true&executor-management-mode=isolation&file-cache=true&generic=false&interface=org.apache.dubbo.demo.DemoService&methods=sayHello,sayHelloAsync&pid=7900&prefer.serialization=fastjson2,hessian2&service-name-mapping=true&side=provider&sticky=false&unloadClusterRelated=false
继续向下看下refreshInvoker(urls)，此方法会将invoker的url转化成Invoker对象
/**
* Convert the invokerURL list to the Invoker Map. The rules of the conversion are as follows:
* <ol>
* <li> If URL has been converted to invoker, it is no longer re-referenced and obtained directly from the cache,
* and notice that any parameter changes in the URL will be re-referenced.</li>
* <li>If the incoming invoker list is not empty, it means that it is the latest invoker list.</li>
* <li>If the list of incoming invokerUrl is empty, It means that the rule is only a override rule or a route
* rule, which needs to be re-contrasted to decide whether to re-reference.</li>
* </ol>
*
* @param invokerUrls this parameter can't be null
*/
private void refreshInvoker(List<URL> invokerUrls) {
Assert.notNull(invokerUrls, "invokerUrls should not be null");
if (invokerUrls.size() == 1
&& invokerUrls.get(0) != null
&& EMPTY_PROTOCOL.equals(invokerUrls.get(0).getProtocol())) {
refreshRouter(
BitList.emptyList(), () -> this.forbidden = true // Forbid to access
);
destroyAllInvokers(); // Close all invokers
} else {
this.forbidden = false; // Allow to access
if (invokerUrls == Collections.<URL>emptyList()) {
invokerUrls = new ArrayList<>();
}
// use local reference to avoid NPE as this.cachedInvokerUrls will be set null by destroyAllInvokers().
Set<URL> localCachedInvokerUrls = this.cachedInvokerUrls;
if (invokerUrls.isEmpty()) {
if (CollectionUtils.isNotEmpty(localCachedInvokerUrls)) {
// 1-4 Empty address.
logger.warn(
REGISTRY_EMPTY_ADDRESS,
"configuration ",
"",
"Service" + serviceKey
+ " received empty address list with no EMPTY protocol set, trigger empty protection.");
invokerUrls.addAll(localCachedInvokerUrls);
}
} else {
// 走这里
localCachedInvokerUrls = new HashSet<>();
localCachedInvokerUrls.addAll(invokerUrls); // Cached invoker urls, convenient for comparison
// 本地缓存invokerUrls赋值
this.cachedInvokerUrls = localCachedInvokerUrls;
}
if (invokerUrls.isEmpty()) {
return;
}
// use local reference to avoid NPE as this.urlInvokerMap will be set null concurrently at
// destroyAllInvokers().
Map<URL, Invoker<T>> localUrlInvokerMap = this.urlInvokerMap;
// can't use local reference as oldUrlInvokerMap's mappings might be removed directly at toInvokers().
Map<URL, Invoker<T>> oldUrlInvokerMap = null;
if (localUrlInvokerMap != null) {
// the initial capacity should be set greater than the maximum number of entries divided by the load
// factor to avoid resizing.
oldUrlInvokerMap =
new LinkedHashMap<>(Math.round(1 + localUrlInvokerMap.size() / DEFAULT_HASHMAP_LOAD_FACTOR));
localUrlInvokerMap.forEach(oldUrlInvokerMap::put);
}
// 转化逻辑入口
Map<URL, Invoker<T>> newUrlInvokerMap =
toInvokers(oldUrlInvokerMap, invokerUrls); // Translate url list to Invoker map
/*
* If the calculation is wrong, it is not processed.
*
* 1. The protocol configured by the client is inconsistent with the protocol of the server.
* eg: consumer protocol = dubbo, provider only has other protocol services(rest).
* 2. The registration center is not robust and pushes illegal specification data.
*
*/
if (CollectionUtils.isEmptyMap(newUrlInvokerMap)) {
// 3-1 - Failed to convert the URL address into Invokers.
logger.error(
PROXY_FAILED_CONVERT_URL,
"inconsistency between the client protocol and the protocol of the server",
"",
"urls to invokers error",
new IllegalStateException("urls to invokers error. invokerUrls.size :" + invokerUrls.size()
+ ", invoker.size :0. urls :" + invokerUrls.toString()));
return;
}
List<Invoker<T>> newInvokers = Collections.unmodifiableList(new ArrayList<>(newUrlInvokerMap.values()));
BitList<Invoker<T>> finalInvokers =
multiGroup ? new BitList<>(toMergeInvokerList(newInvokers)) : new BitList<>(newInvokers);
// pre-route and build cache
refreshRouter(finalInvokers.clone(), () -> this.setInvokers(finalInvokers));
this.urlInvokerMap = newUrlInvokerMap;
try {
destroyUnusedInvokers(oldUrlInvokerMap, newUrlInvokerMap); // Close the unused Invoker
} catch (Exception e) {
logger.warn(REGISTRY_FAILED_DESTROY_SERVICE, "", "", "destroyUnusedInvokers error. ", e);
}
// notify invokers refreshed
this.invokersChanged();
}
}
调试可以发现，只有category为providers时，才会将invokerUrl进行转化
首先，会对cachedInvokerUrls进行赋值，此时的Map<URL, Invoker<T>> localUrlInvokerMap = this.urlInvokerMap还是null，也就是说，此时本地对invokerMap的缓存还是空，接下来执行关键方法toInvokers，此方法会将invokerUrl list转化为Invoker Map。
/**
* Turn urls into invokers, and if url has been referred, will not re-reference.
* the items that will be put into newUrlInvokeMap will be removed from oldUrlInvokerMap.
* @param oldUrlInvokerMap it might be modified during the process.
* @param urls
* @return invokers
*/
private Map<URL, Invoker<T>> toInvokers(Map<URL, Invoker<T>> oldUrlInvokerMap, List<URL> urls) {
Map<URL, Invoker<T>> newUrlInvokerMap =
new ConcurrentHashMap<>(urls == null ? 1 : (int) (urls.size() / 0.75f + 1));
if (urls == null || urls.isEmpty()) {
return newUrlInvokerMap;
}
String queryProtocols = this.queryMap.get(PROTOCOL_KEY);
for (URL providerUrl : urls) {
if (!checkProtocolValid(queryProtocols, providerUrl)) {
continue;
}
URL url = mergeUrl(providerUrl);
// Cache key is url that does not merge with consumer side parameters,
// regardless of how the consumer combines parameters,
// if the server url changes, then refer again
Invoker<T> invoker = oldUrlInvokerMap == null ? null : oldUrlInvokerMap.remove(url);
if (invoker == null) { // Not in the cache, refer again
try {
boolean enabled = true;
if (url.hasParameter(DISABLED_KEY)) {
enabled = !url.getParameter(DISABLED_KEY, false);
} else {
enabled = url.getParameter(ENABLED_KEY, true);
}
if (enabled) {
// 这里是核心，走到DubboProtocol中的refer方法
invoker = protocol.refer(serviceType, url);
}
} catch (Throwable t) {
// Thrown by AbstractProtocol.optimizeSerialization()
if (t instanceof RpcException && t.getMessage().contains("serialization optimizer")) {
// 4-2 - serialization optimizer class initialization failed.
logger.error(
PROTOCOL_FAILED_INIT_SERIALIZATION_OPTIMIZER,
"typo in optimizer class",
"",
"Failed to refer invoker for interface:" + serviceType + ",url:(" + url + ")"
+ t.getMessage(),
t);
} else {
// 4-3 - Failed to refer invoker by other reason.
logger.error(
PROTOCOL_FAILED_REFER_INVOKER,
"",
"",
"Failed to refer invoker for interface:" + serviceType + ",url:(" + url + ")"
+ t.getMessage(),
t);
}
}
if (invoker != null) { // Put new invoker in cache
newUrlInvokerMap.put(url, invoker);
}
} else {
newUrlInvokerMap.put(url, invoker);
}
}
return newUrlInvokerMap;
}
在此方法中，核心看一下invoker = protocol.refer(serviceType, url);调用DubboProtocol下的refer方法去获取Invoker。
@Override
public <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException {
checkDestroyed();
return protocolBindingRefer(type, url);
}
@Override
public <T> Invoker<T> protocolBindingRefer(Class<T> serviceType, URL url) throws RpcException {
checkDestroyed();
optimizeSerialization(url);
// create rpc invoker.
DubboInvoker<T> invoker = new DubboInvoker<>(serviceType, url, getClients(url), invokers);
invokers.add(invoker);
return invoker;
}
如上，创建DubboInvoker时，会getClient(url)，继续向下看
private ClientsProvider getClients(URL url) {
int connections = url.getParameter(CONNECTIONS_KEY, 0);
// whether to share connection
// if not configured, connection is shared, otherwise, one connection for one service
if (connections == 0) {
/*
* The xml configuration should have a higher priority than properties.
*/
String shareConnectionsStr = StringUtils.isBlank(url.getParameter(SHARE_CONNECTIONS_KEY, (String) null))
? ConfigurationUtils.getProperty(
url.getOrDefaultApplicationModel(), SHARE_CONNECTIONS_KEY, DEFAULT_SHARE_CONNECTIONS)
: url.getParameter(SHARE_CONNECTIONS_KEY, (String) null);
connections = Integer.parseInt(shareConnectionsStr);
// 走到这里
return getSharedClient(url, connections);
}
List<ExchangeClient> clients =
IntStream.range(0, connections).mapToObj((i) -> initClient(url)).collect(Collectors.toList());
return new ExclusiveClientsProvider(clients);
}
再向下走getSharedClient
/**
* Get shared connection
* @param url
* @param connectNum connectNum must be greater than or equal to 1
*/
@SuppressWarnings("unchecked")
private SharedClientsProvider getSharedClient(URL url, int connectNum) {
String key = url.getAddress();
// connectNum must be greater than or equal to 1
int expectedConnectNum = Math.max(connectNum, 1);
return referenceClientMap.compute(key, (originKey, originValue) -> {
if (originValue != null && originValue.increaseCount()) {
return originValue;
} else {
return new SharedClientsProvider(
this, originKey, buildReferenceCountExchangeClientList(url, expectedConnectNum));
}
});
}
这里会获取dubbo接口provider的address，并且获取连接数，默认是1
之后，执行buildReferenceCountExchangeClientList批量构建客户端
// 批量构建客户端
private List<ReferenceCountExchangeClient> buildReferenceCountExchangeClientList(URL url, int connectNum) {
List<ReferenceCountExchangeClient> clients = new ArrayList<>();
for (int i = 0; i < connectNum; i++) {
clients.add(buildReferenceCountExchangeClient(url));
}
return clients;
}
/**
* Build a single client 构建一个客户端
* @param url
* @return
*/
private ReferenceCountExchangeClient buildReferenceCountExchangeClient(URL url) {
ExchangeClient exchangeClient = initClient(url);
ReferenceCountExchangeClient client = new ReferenceCountExchangeClient(exchangeClient, DubboCodec.NAME);
// read configs
int shutdownTimeout = ConfigurationUtils.getServerShutdownTimeout(url.getScopeModel());
client.setShutdownWaitTime(shutdownTimeout);
return client;
}
构建客户端之前会初始化一个客户端，执行initClient(url)
/**
* 创建一个新的网络连接
*/
private ExchangeClient initClient(URL url) {
/*
* Instance of url is InstanceAddressURL, so addParameter actually adds parameters into ServiceInstance,
* which means params are shared among different services. Since client is shared among services this is currently not a problem.
*/
String str = url.getParameter(CLIENT_KEY, url.getParameter(SERVER_KEY, DEFAULT_REMOTING_CLIENT));
// 检测通信模型是否是BIO，若是，则直接异常
if (StringUtils.isNotEmpty(str)
&& !url.getOrDefaultFrameworkModel()
.getExtensionLoader(Transporter.class)
.hasExtension(str)) {
throw new RpcException("Unsupported client type: " + str + "," + " supported client type is "
+ StringUtils.join(
url.getOrDefaultFrameworkModel()
.getExtensionLoader(Transporter.class)
.getSupportedExtensions(),
" "));
}
try {
ScopeModel scopeModel = url.getScopeModel();
int heartbeat = UrlUtils.getHeartbeat(url);
// Replace InstanceAddressURL with ServiceConfigURL.
url = new ServiceConfigURL(
DubboCodec.NAME,
url.getUsername(),
url.getPassword(),
url.getHost(),
url.getPort(),
url.getPath(),
url.getAllParameters());
url = url.addParameter(CODEC_KEY, DubboCodec.NAME);
// enable heartbeat by default
url = url.addParameterIfAbsent(HEARTBEAT_KEY, Integer.toString(heartbeat));
url = url.setScopeModel(scopeModel);
// connection should be lazy
return url.getParameter(LAZY_CONNECT_KEY, false)
? new LazyConnectExchangeClient(url, requestHandler)
: Exchangers.connect(url, requestHandler);
} catch (RemotingException e) {
throw new RpcException("Fail to create remoting client for service(" + url + "): " + e.getMessage(), e);
}
}
init过程，会先得到CLIENT_KEY，此处是netty，再去校验，若是BIO（阻塞式IO，有严重性能问题），则抛异常。
然后对url进行处理，得到最后要建立网络连接的url，再去判断是否以lazy模式去连接。此处并不以lazy进行加载，直接走Exchangers.connect
连接的url示例如下
dubbo://10.1.xx.219:20880/org.apache.dubbo.demo.DemoService?application=dubbo-demo-api-consumer&background=false&category=providers,configurators,routers&check=false&codec=dubbo&deprecated=false&dubbo=2.0.2&dynamic=true&executor-management-mode=isolation&file-cache=true&generic=true&heartbeat=60000&interface=org.apache.dubbo.demo.DemoService&methods=sayHello,sayHelloAsync&pid=25608&prefer.serialization=fastjson2,hessian2&service-name-mapping=true&side=consumer&sticky=false&unloadClusterRelated=false
Exchangers.connect的具体流程上一篇文章，服务发布与注册时讲到了，就是使用Transporters去进行netty网络连接，依靠Exchanger进行数据交换。此处不再展开。
到目前，已经与服务的provider建立了网络通信,回到前面的创建DubboInvoker
DubboInvoker<T> invoker = new DubboInvoker<>(serviceType, url, getClients(url), invokers)
创建好的DubboInvoker示例如下图
回到前面的toInvokers()，会将invoker塞进newUrlInvokerMap
接着跳回org.apache.dubbo.registry.integration.RegistryDirectory#refreshInvoker
// newInvokers就是toInvoker转化得到的invoker列表
List<Invoker<T>> newInvokers = Collections.unmodifiableList(new ArrayList<>(newUrlInvokerMap.values()));
BitList<Invoker<T>> finalInvokers =
multiGroup ? new BitList<>(toMergeInvokerList(newInvokers)) : new BitList<>(newInvokers);
// pre-route and build cache 这里会填充本地缓存invokers
refreshRouter(finalInvokers.clone(), () -> this.setInvokers(finalInvokers));
this.urlInvokerMap = newUrlInvokerMap;
try {
destroyUnusedInvokers(oldUrlInvokerMap, newUrlInvokerMap); // Close the unused Invoker
} catch (Exception e) {
logger.warn(REGISTRY_FAILED_DESTROY_SERVICE, "", "", "destroyUnusedInvokers error. ", e);
}
// notify invokers refreshed
this.invokersChanged();
这里将toInvoker得到的newInvokers和newUrlInvokermap放到本地缓存this.invokers和this.urlInvokerMap中（后面进行服务调用时，用到的Invoker就是从缓存中取的）。最后，将invokers改变的消息进行通知，按照规则刷新invokers并发布事件，这个notify过程不再展开。
一路return，最后，将包含着得到invoker相关缓存的RegistryDirectory封装成ClusterInvoker进行返回得到ScopeClusterInvoker，如下图所示
至此，接口级Invoker刷新就完成了，主要是订阅了provider节点，拿到服务url list，并根据url list构建服务的Invoker，填充本地Invoker相关缓存。
回到org.apache.dubbo.registry.client.migration.MigrationInvoker#migrateToApplicationFirstInvoker
@Override
public void migrateToApplicationFirstInvoker(MigrationRule newRule) {
CountDownLatch latch = new CountDownLatch(0);
refreshInterfaceInvoker(latch);
refreshServiceDiscoveryInvoker(latch);
// directly calculate preferred invoker, will not wait until address notify
// calculation will re-occurred when address notify later
calcPreferredInvoker(newRule);
}
处理应用级Invoker
下一步是执行应用级Invoker的刷新
从refreshServiceDiscoveryInvoker走到doCreateInvoker
protected <T> ClusterInvoker<T> doCreateInvoker(
DynamicDirectory<T> directory, Cluster cluster, Registry registry, Class<T> type) {
directory.setRegistry(registry);
directory.setProtocol(protocol);
// all attributes of REFER_KEY
Map<String, String> parameters =
new HashMap<>(directory.getConsumerUrl().getParameters());
URL urlToRegistry = new ServiceConfigURL(
parameters.get(PROTOCOL_KEY) == null ? CONSUMER : parameters.get(PROTOCOL_KEY),
parameters.remove(REGISTER_IP_KEY),
0,
getPath(parameters, type),
parameters);
urlToRegistry = urlToRegistry.setScopeModel(directory.getConsumerUrl().getScopeModel());
urlToRegistry = urlToRegistry.setServiceModel(directory.getConsumerUrl().getServiceModel());
if (directory.isShouldRegister()) {
directory.setRegisteredConsumerUrl(urlToRegistry);
registry.register(directory.getRegisteredConsumerUrl());
}
directory.buildRouterChain(urlToRegistry);
directory.subscribe(toSubscribeUrl(urlToRegistry));
return (ClusterInvoker<T>) cluster.join(directory, true);
}
directory和registry类型如图所示
此时，ServiceDiscoveryRegistry是不去向zk注册节点的，因此，我们可以跳过register直接看订阅的过程。
我们直接来到ServiceDiscoveryRegistry的doSubscribe方法，看看它在订阅时都干了什么。
@Override
public void doSubscribe(URL url, NotifyListener listener) {
url = addRegistryClusterKey(url);
serviceDiscovery.subscribe(url, listener);
Set<String> mappingByUrl = ServiceNameMapping.getMappingByUrl(url);
String key = ServiceNameMapping.buildMappingKey(url);
if (mappingByUrl == null) {
Lock mappingLock = serviceNameMapping.getMappingLock(key);
try {
mappingLock.lock();
// 从本地缓存中获得应用-接口映射关系列表
mappingByUrl = serviceNameMapping.getMapping(url);
try {
MappingListener mappingListener = new DefaultMappingListener(url, mappingByUrl, listener);
mappingByUrl = serviceNameMapping.getAndListen(this.getUrl(), url, mappingListener);
synchronized (mappingListeners) {
mappingListeners
.computeIfAbsent(url.getProtocolServiceKey(), (k) -> new ConcurrentHashSet<>())
.add(mappingListener);
}
} catch (Exception e) {
logger.warn(
INTERNAL_ERROR,
"",
"",
"Cannot find app mapping for service " + url.getServiceInterface() + ", will not migrate.",
e);
}
if (CollectionUtils.isEmpty(mappingByUrl)) {
logger.info(
"No interface-apps mapping found in local cache, stop subscribing, will automatically wait for mapping listener callback: "
+ url);
return;
}
} finally {
mappingLock.unlock();
}
}
subscribeURLs(url, listener, mappingByUrl);
}
首先进入到serviceDiscovery.subscribe(url, listener);
@Override
public void subscribe(URL url, NotifyListener listener) {
metadataInfo.addSubscribedURL(url);
}
向元数据添加了已订阅的url，填充subscribedServiceURLs缓存
具体操作如下图
接着，中间处理Listener和mappingByUrl
映射及元数据获取
这里看下mappingByUrl = serviceNameMapping.getMapping(url);，会根据url中的provider-by来获取映射，若获取不到，则从本地缓存中获取。
在这之前，有一个初始化mapping的操作，在ServiceDiscoveryRegistry的构造方法中，一路向下走，会去init映射关系
public ServiceDiscoveryRegistry(URL registryURL, ApplicationModel applicationModel) {
super(registryURL);
this.serviceDiscovery = createServiceDiscovery(registryURL);
// 这里一路向下调试，会有一个init方法
this.serviceNameMapping =
(AbstractServiceNameMapping) ServiceNameMapping.getDefaultExtension(registryURL.getScopeModel());
super.applicationModel = applicationModel;
}
此处用到了spi扩展机制，可以在下面构造方法打断点
public MappingCacheManager(boolean enableFileCache, String name, ScheduledExecutorService executorService) {
String filePath = System.getProperty("dubbo.mapping.cache.filePath");
String fileName = System.getProperty("dubbo.mapping.cache.fileName");
if (StringUtils.isEmpty(fileName)) {
fileName = DEFAULT_FILE_NAME;
}
if (StringUtils.isNotEmpty(name)) {
fileName = fileName + "." + name;
}
String rawEntrySize = System.getProperty("dubbo.mapping.cache.entrySize");
int entrySize = StringUtils.parseInteger(rawEntrySize);
entrySize = (entrySize == 0 ? DEFAULT_ENTRY_SIZE : entrySize);
String rawMaxFileSize = System.getProperty("dubbo.mapping.cache.maxFileSize");
long maxFileSize = StringUtils.parseLong(rawMaxFileSize);
// 就是这里了
init(enableFileCache, filePath, fileName, entrySize, maxFileSize, 50, executorService);
}
下面看下init方法
protected void init(
boolean enableFileCache,
String filePath,
String fileName,
int entrySize,
long fileSize,
int interval,
ScheduledExecutorService executorService) {
this.cache = new LRUCache<>(entrySize);
try {
cacheStore = FileCacheStoreFactory.getInstance(filePath, fileName, enableFileCache);
Map<String, String> properties = cacheStore.loadCache(entrySize);
if (logger.isDebugEnabled()) {
logger.debug("Successfully loaded " + getName() + " cache from file " + fileName + ", entries "
+ properties.size());
}
for (Map.Entry<String, String> entry : properties.entrySet()) {
String key = entry.getKey();
String value = entry.getValue();
this.cache.put(key, toValueType(value));
}
// executorService can be empty if FileCacheStore fails
if (executorService == null) {
this.executorService = Executors.newSingleThreadScheduledExecutor(
new NamedThreadFactory("Dubbo-cache-refreshing-scheduler", true));
} else {
this.executorService = executorService;
}
this.executorService.scheduleWithFixedDelay(
new CacheRefreshTask<>(this.cacheStore, this.cache, this, fileSize),
10,
interval,
TimeUnit.MINUTES);
} catch (Exception e) {
logger.error(COMMON_FAILED_LOAD_MAPPING_CACHE, "", "", "Load mapping from local cache file error ", e);
}
}
在这里进行初始化，从本地文件中取映射关系（consumer会自己在本地维护一个映射关系）
我们回到mappingByUrl = serviceNameMapping.getMapping(url);，会获取映射关系。再往下走，看一下
mappingByUrl = serviceNameMapping.getAndListen(this.getUrl(), url, mappingListener)
@Override
public Set<String> getAndListen(URL registryURL, URL subscribedURL, MappingListener listener) {
String key = ServiceNameMapping.buildMappingKey(subscribedURL);
// use previously cached services.
Set<String> mappingServices = mappingCacheManager.get(key);
// Asynchronously register listener in case previous cache does not exist or cache expired.
if (CollectionUtils.isEmpty(mappingServices)) {
// 若本地缓存中没有，则从zk获取
try {
logger.info("Local cache mapping is empty");
// 这里是核心
mappingServices = (new AsyncMappingTask(listener, subscribedURL, false)).call();
} catch (Exception e) {
// ignore
}
if (CollectionUtils.isEmpty(mappingServices)) {
String registryServices = registryURL.getParameter(SUBSCRIBED_SERVICE_NAMES_KEY);
if (StringUtils.isNotEmpty(registryServices)) {
logger.info(subscribedURL.getServiceInterface() + " mapping to " + registryServices
+ " instructed by registry subscribed-services.");
mappingServices = parseServices(registryServices);
}
}
if (CollectionUtils.isNotEmpty(mappingServices)) {
this.putCachedMapping(ServiceNameMapping.buildMappingKey(subscribedURL), mappingServices);
}
} else {
// 若本地缓存存在，则向下执行异步任务
ExecutorService executorService = applicationModel
.getFrameworkModel()
.getBeanFactory()
.getBean(FrameworkExecutorRepository.class)
.getMappingRefreshingExecutor();
executorService.submit(new AsyncMappingTask(listener, subscribedURL, true));
}
return mappingServices;
}
上述方法中，若本地缓存中存在mappingServices，则会继续执行subscribeURLs(url, listener, mappingByUrl)。
若本地缓存不存在，走call()流程
@Override
public Set<String> call() throws Exception {
synchronized (mappingListeners) {
Set<String> mappedServices = emptySet();
try {
String mappingKey = ServiceNameMapping.buildMappingKey(subscribedURL);
if (listener != null) {
// 重点看下这里
mappedServices = toTreeSet(getAndListen(subscribedURL, listener));
Set<MappingListener> listeners =
mappingListeners.computeIfAbsent(mappingKey, _k -> new HashSet<>());
listeners.add(listener);
if (CollectionUtils.isNotEmpty(mappedServices)) {
if (notifyAtFirstTime) {
// guarantee at-least-once notification no matter what kind of underlying meta server is
// used.
// listener notification will also cause updating of mapping cache.
listener.onEvent(new MappingChangedEvent(mappingKey, mappedServices));
}
}
} else {
mappedServices = get(subscribedURL);
if (CollectionUtils.isNotEmpty(mappedServices)) {
AbstractServiceNameMapping.this.putCachedMapping(mappingKey, mappedServices);
}
}
} catch (Exception e) {
logger.error(
COMMON_FAILED_LOAD_MAPPING_CACHE,
"",
"",
"Failed getting mapping info from remote center. ",
e);
}
return mappedServices;
}
}
}
call()流程，会去获取mappingServices，然后执行一个listener.onEvent(new MappingChangedEvent(mappingKey, mappedServices));，去触发一个本地缓存修改事件，修改本地缓存中的映射信息。
下面看下call()方法中的getAndListen(subscribedURL, listener)
org.apache.dubbo.registry.client.metadata.MetadataServiceNameMapping#getAndListen
@Override
public Set<String> getAndListen(URL url, MappingListener mappingListener) {
String serviceInterface = url.getServiceInterface();
// randomly pick one metadata report is ok for it's guaranteed all metadata report will have the same mapping
// data.
String registryCluster = getRegistryCluster(url);
MetadataReport metadataReport = metadataReportInstance.getMetadataReport(registryCluster);
if (metadataReport == null) {
return Collections.emptySet();
}
return metadataReport.getServiceAppMapping(serviceInterface, mappingListener, url);
}
看最后的getServiceAppMapping方法
@Override
public Set<String> getServiceAppMapping(String serviceKey, MappingListener listener, URL url) {
String path = buildPathKey(DEFAULT_MAPPING_GROUP, serviceKey);
MappingDataListener mappingDataListener = ConcurrentHashMapUtils.computeIfAbsent(casListenerMap, path, _k -> {
MappingDataListener newMappingListener = new MappingDataListener(serviceKey, path);
zkClient.addDataListener(path, newMappingListener);
return newMappingListener;
});
mappingDataListener.addListener(listener);
return getAppNames(zkClient.getContent(path));
}
这里，可以看到，使用zkClient去获取了应用名称列表。获取到映射关系之后，回到下面的逻辑里继续往下进行。
接着，执行subscribeURLs(url, listener, mappingByUrl)
protected void subscribeURLs(URL url, NotifyListener listener, Set<String> serviceNames) {
serviceNames = toTreeSet(serviceNames);
String serviceNamesKey = toStringKeys(serviceNames);
String serviceKey = url.getServiceKey();
logger.info(
String.format("Trying to subscribe from apps %s for service key %s, ", serviceNamesKey, serviceKey));
// register ServiceInstancesChangedListener
Lock appSubscriptionLock = getAppSubscription(serviceNamesKey);
try {
appSubscriptionLock.lock();
ServiceInstancesChangedListener serviceInstancesChangedListener = serviceListeners.get(serviceNamesKey);
if (serviceInstancesChangedListener == null) {
serviceInstancesChangedListener = serviceDiscovery.createListener(serviceNames);
// 循环获取每个应用下的服务实例列表
for (String serviceName : serviceNames) {
// 使用curator去获取服务实例
List<ServiceInstance> serviceInstances = serviceDiscovery.getInstances(serviceName);
// 服务实例不为空，则去执行服务实例更新事件
if (CollectionUtils.isNotEmpty(serviceInstances)) {
serviceInstancesChangedListener.onEvent(
new ServiceInstancesChangedEvent(serviceName, serviceInstances));
}
}
serviceListeners.put(serviceNamesKey, serviceInstancesChangedListener);
}
if (!serviceInstancesChangedListener.isDestroyed()) {
listener.addServiceListener(serviceInstancesChangedListener);
serviceInstancesChangedListener.addListenerAndNotify(url, listener);
ServiceInstancesChangedListener finalServiceInstancesChangedListener = serviceInstancesChangedListener;
String serviceDiscoveryName =
url.getParameter(RegistryConstants.REGISTRY_CLUSTER_KEY, url.getProtocol());
MetricsEventBus.post(
RegistryEvent.toSsEvent(
url.getApplicationModel(), serviceKey, Collections.singletonList(serviceDiscoveryName)),
() -> {
serviceDiscovery.addServiceInstancesChangedListener(finalServiceInstancesChangedListener);
return null;
});
} else {
logger.info(String.format("Listener of %s has been destroyed by another thread.", serviceNamesKey));
serviceListeners.remove(serviceNamesKey);
}
} finally {
appSubscriptionLock.unlock();
}
}
这里会根据serviceNames（应用名称列表），获取每个Application下的服务实例信息列表，之后，若服务实例存在，则会去执行服务实例更新的事件。
下面先来看下onEvent方法下的doOnEvent中获取MetadataInfo的逻辑
// get MetadataInfo with revision
for (Map.Entry<String, List<ServiceInstance>> entry : revisionToInstances.entrySet()) {
String revision = entry.getKey();
List<ServiceInstance> subInstances = entry.getValue();
MetadataInfo metadata = subInstances.stream()
.map(ServiceInstance::getServiceMetadata)
.filter(Objects::nonNull)
.filter(m -> revision.equals(m.getRevision()))
.findFirst()
.orElseGet(() -> serviceDiscovery.getRemoteMetadata(revision, subInstances));
parseMetadata(revision, metadata, localServiceToRevisions);
// update metadata into each instance, in case new instance created.
for (ServiceInstance tmpInstance : subInstances) {
MetadataInfo originMetadata = tmpInstance.getServiceMetadata();
if (originMetadata == null || !Objects.equals(originMetadata.getRevision(), metadata.getRevision())) {
tmpInstance.setServiceMetadata(metadata);
}
}
}
其中，看下获取远程元数据的getRomteMetadata方法
@Override
public MetadataInfo getRemoteMetadata(String revision, List<ServiceInstance> instances) {
MetadataInfo metadata = metaCacheManager.get(revision);
if (metadata != null && metadata != MetadataInfo.EMPTY) {
// 缓存中有，则初始化
metadata.init();
// metadata loaded from cache
if (logger.isDebugEnabled()) {
logger.debug("MetadataInfo for revision=" + revision + ", " + metadata);
}
return metadata;
}
synchronized (metaCacheManager) {
// try to load metadata from remote.
int triedTimes = 0;
// 重试机制
while (triedTimes < 3) {
metadata = MetricsEventBus.post(
MetadataEvent.toSubscribeEvent(applicationModel),
// 这里，继续向下看
() -> MetadataUtils.getRemoteMetadata(revision, instances, metadataReport),
result -> result != MetadataInfo.EMPTY);
if (metadata != MetadataInfo.EMPTY) { // succeeded
metadata.init();
break;
} else { // failed
if (triedTimes > 0) {
if (logger.isDebugEnabled()) {
logger.debug("Retry the " + triedTimes + " times to get metadata for revision=" + revision);
}
}
triedTimes++;
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
}
}
}
if (metadata == MetadataInfo.EMPTY) {
logger.error(
REGISTRY_FAILED_LOAD_METADATA,
"",
"",
"Failed to get metadata for revision after 3 retries, revision=" + revision);
} else {
metaCacheManager.put(revision, metadata);
}
}
return metadata;
}
在这里想吐槽下，链路是真的深，继续向下看MetadataUtils.getRemoteMetadata
public static MetadataInfo getRemoteMetadata(
String revision, List<ServiceInstance> instances, MetadataReport metadataReport) {
ServiceInstance instance = selectInstance(instances);
String metadataType = ServiceInstanceMetadataUtils.getMetadataStorageType(instance);
MetadataInfo metadataInfo;
try {
if (logger.isDebugEnabled()) {
logger.debug("Instance " + instance.getAddress() + " is using metadata type " + metadataType);
}
// 从远程获取
if (REMOTE_METADATA_STORAGE_TYPE.equals(metadataType)) {
metadataInfo = MetadataUtils.getMetadata(revision, instance, metadataReport);
} else {
// 通过rpc，从本地获取
// change the instance used to communicate to avoid all requests route to the same instance
ProxyHolder proxyHolder = null;
try {
proxyHolder = MetadataUtils.referProxy(instance);
metadataInfo = proxyHolder
.getProxy()
.getMetadataInfo(ServiceInstanceMetadataUtils.getExportedServicesRevision(instance));
} finally {
MetadataUtils.destroyProxy(proxyHolder);
}
}
} catch (Exception e) {
logger.error(
REGISTRY_FAILED_LOAD_METADATA,
"",
"",
"Failed to get app metadata for revision " + revision + " for type " + metadataType
+ " from instance " + instance.getAddress(),
e);
metadataInfo = null;
}
if (metadataInfo == null) {
metadataInfo = MetadataInfo.EMPTY;
}
return metadataInfo;
}
这里通过jvm参数，可以配置从provider本地获取还是从远程zk获取元数据。
从zk远程获取，会走到下面这个方法
@Override
public MetadataInfo getAppMetadata(SubscriberMetadataIdentifier identifier, Map<String, String> instanceMetadata) {
String content = zkClient.getContent(getNodePath(identifier));
return JsonUtils.toJavaObject(content, MetadataInfo.class);
}
从本地获取，会通过一个代理，去provider服务器获取到元数据
在proxyHolder = MetadataUtils.referProxy(instance)中，referProxy方法中有下面一段代码
Invoker<MetadataService> invoker = protocol.refer(MetadataService.class, url);
还是走到DubboProtocol中，通过netty建立通信，最终获取目标元数据。
到这里，元数据获取就结束了
下面，让我们回到org.apache.dubbo.registry.client.ServiceDiscoveryRegistry#subscribeURLs
接着看serviceInstancesChangedListener.addListenerAndNotify(url, listener);
public synchronized void addListenerAndNotify(URL url, NotifyListener listener) {
if (destroyed.get()) {
return;
}
Set<NotifyListenerWithKey> notifyListeners =
this.listeners.computeIfAbsent(url.getServiceKey(), _k -> new ConcurrentHashSet<>());
String protocol = listener.getConsumerUrl().getParameter(PROTOCOL_KEY, url.getProtocol());
ProtocolServiceKey protocolServiceKey = new ProtocolServiceKey(
url.getServiceInterface(),
url.getVersion(),
url.getGroup(),
!CommonConstants.CONSUMER.equals(protocol) ? protocol : null);
NotifyListenerWithKey listenerWithKey = new NotifyListenerWithKey(protocolServiceKey, listener);
notifyListeners.add(listenerWithKey);
// Aggregate address and notify on subscription.
List<URL> urls = getAddresses(protocolServiceKey, listener.getConsumerUrl());
if (CollectionUtils.isNotEmpty(urls)) {
logger.info(String.format(
"Notify serviceKey: %s, listener: %s with %s urls on subscription",
protocolServiceKey, listener, urls.size()));
listener.notify(urls);
}
}
List<URL> urls = getAddresses(protocolServiceKey, listener.getConsumerUrl())通过此方法聚合得到的url为InstanceAddressUrl（这个方法里用到了前面从远程获取MetadataInfo的本地缓存信息，可以自己看一下具体实现），示例如下
DefaultServiceInstance{serviceName='dubbo-demo-api-provider', host='10.1.xx.219', port=20880, enabled=true, healthy=true, metadata={dubbo.endpoints=[{"port":20880,"protocol":"dubbo"}], dubbo.metadata-service.url-params={"prefer.serialization":"fastjson2,hessian2","version":"1.0.0","dubbo":"2.0.2","side":"provider","port":"20880","protocol":"dubbo"}, dubbo.metadata.revision=341cd1d1e10ad64119c07e88ad034887, dubbo.metadata.storage-type=local, timestamp=1742004299061}}
之后，会对urls进行notify，根据notify一路跳到下面这个方法org.apache.dubbo.registry.client.ServiceDiscoveryRegistryDirectory#refreshInvoker
private void refreshInvoker(List<URL> invokerUrls) {
Assert.notNull(invokerUrls, "invokerUrls should not be null, use EMPTY url to clear current addresses.");
this.originalUrls = invokerUrls;
if (invokerUrls.size() == 1 && EMPTY_PROTOCOL.equals(invokerUrls.get(0).getProtocol())) {
refreshRouter(
BitList.emptyList(), () -> this.forbidden = true // Forbid to access
);
destroyAllInvokers(); // Close all invokers
} else {
this.forbidden = false; // Allow accessing
if (CollectionUtils.isEmpty(invokerUrls)) {
return;
}
// use local reference to avoid NPE as this.urlInvokerMap will be set null concurrently at
// destroyAllInvokers().
Map<ProtocolServiceKeyWithAddress, Invoker<T>> localUrlInvokerMap = this.urlInvokerMap;
// can't use local reference as oldUrlInvokerMap's mappings might be removed directly at toInvokers().
Map<ProtocolServiceKeyWithAddress, Invoker<T>> oldUrlInvokerMap = null;
if (localUrlInvokerMap != null) {
// the initial capacity should be set greater than the maximum number of entries divided by the load
// factor to avoid resizing.
oldUrlInvokerMap =
new LinkedHashMap<>(Math.round(1 + localUrlInvokerMap.size() / DEFAULT_HASHMAP_LOAD_FACTOR));
localUrlInvokerMap.forEach(oldUrlInvokerMap::put);
}
// 核心在这，将url转化成Invoker
Map<ProtocolServiceKeyWithAddress, Invoker<T>> newUrlInvokerMap =
toInvokers(oldUrlInvokerMap, invokerUrls); // Translate url list to Invoker map
logger.info(String.format("Refreshed invoker size %s from registry %s", newUrlInvokerMap.size(), this));
if (CollectionUtils.isEmptyMap(newUrlInvokerMap)) {
logger.error(
PROTOCOL_UNSUPPORTED,
"",
"",
"Unsupported protocol.",
new IllegalStateException(String.format(
"Cannot create invokers from url address list (total %s)", invokerUrls.size())));
return;
}
List<Invoker<T>> newInvokers = Collections.unmodifiableList(new ArrayList<>(newUrlInvokerMap.values()));
BitList<Invoker<T>> finalInvokers =
multiGroup ? new BitList<>(toMergeInvokerList(newInvokers)) : new BitList<>(newInvokers);
// pre-route and build cache
refreshRouter(finalInvokers.clone(), () -> this.setInvokers(finalInvokers));
this.urlInvokerMap = newUrlInvokerMap;
if (oldUrlInvokerMap != null) {
try {
destroyUnusedInvokers(oldUrlInvokerMap, newUrlInvokerMap); // Close the unused Invoker
} catch (Exception e) {
logger.warn(PROTOCOL_FAILED_DESTROY_INVOKER, "", "", "destroyUnusedInvokers error. ", e);
}
}
}
// notify invokers refreshed
this.invokersChanged();
}
这段代码相信已经比较熟悉了，和前面接口级服务发现干的事基本一致，只不过不在一个类中。具体流程在此就不再重复叙述，最后也是创建了netty客户端，然后封装在了一个DubboInvoker中，再将得到的invoker进行缓存填充，最后层层封装，得到ScopeClusterInvoker
再回到下面的方法
@Override
public void migrateToApplicationFirstInvoker(MigrationRule newRule) {
CountDownLatch latch = new CountDownLatch(0);
refreshInterfaceInvoker(latch);
refreshServiceDiscoveryInvoker(latch);
// directly calculate preferred invoker, will not wait until address notify
// calculation will re-occurred when address notify later
calcPreferredInvoker(newRule);
}
接着执行calcPreferredInvoker去从invoker（接口级Invoker）和serviceDiscoveryInvoker（应用级Invoker）中选出currentAvailableInvoker
this.currentAvailableInvoker = serviceDiscoveryInvoker;
这里直接说结果，Invoker平滑迁移机制选出的是serviceDiscoveryInvoker。
下面放下接口级Invoker和应用级Invoker中Directory中urlInvokerMap缓存的区别
接口级Invoker下Directory为RegistryDirectory，其中的urlInvokerMap如下
/**java
* Map<url, Invoker> cache service url to invoker mapping.
* The initial value is null and the midway may be assigned to null, please use the local variable reference
*/
protected volatile Map<URL, Invoker<T>> urlInvokerMap;
应用级Invoker下Directory为ServiceDiscoveryRegistryDirectory，其中的urlInvokerMap如下
/**java
* instance address to invoker mapping.
* The initial value is null and the midway may be assigned to null, please use the local variable reference
*/
private volatile Map<ProtocolServiceKeyWithAddress, Invoker<T>> urlInvokerMap;
别的不再多讲，回到前面选出currentAvailableInvoker
之后一路跳回到org.apache.dubbo.config.ReferenceConfig#createInvoker
最终拿到的invoker是MigrationInvoker，并将其放到ReferenceConfig的invoker缓存中，其中包含了三种invoker，如下图
服务发现流程到这里也就结束了，后续会根据invoker来进行服务调用。
总结
Dubbo3.2.6的服务发现机制是基于Invoker平滑迁移机制来实现的，即在接口级服务发现模型向应用级服务发现模型升级过程中，保证两种类型都兼容的机制。
首先是一系列配置的初始化
之后开始invoker迁移流程
Migration机制中，先对接口级Invoker进行refresh处理，其中先在zk注册consumer节点，然后对providers进行订阅，在订阅时，获取服务提供者节点下的子节点，对接口信息进行保存，另外，会根据url和服务提供者建立netty网络Client，封装成为DubboInvoker，并将这些Invoker放入到RegistryDirectory的本地缓存中，封装得到最后的ScopeClusterInvoker。
接着对应用级的Invoker进行refresh处理，然后对包含目标接口的应用进行订阅，订阅时，首先会获取service-app对应关系，之后通过映射关系，去获取到元数据，之后会处理得到instanceAddressURL，去和服务提供者建立netty网络Client，也封装成DubboInvoker，并放入到ServiceDiscoveryRegistryDirectory的本地缓存中，封装得到最后的ScopeClusterInvoker。
之后，对二者进行选举，选出当前可用的invoker，即应用级Invoker（serviceDiscoveryInvoker）
最后统一封装进ReferenceConfig的MigrationInvoker本地缓存中。
