欢迎大家关注 github.com/hsfxuebao ，希望对大家有所帮助，要是觉得可以的话麻烦给点一下Star哈
1. consumer的invoke发送请求
先看一些整体的流程图：
发送请求时，经过各种过滤器，过滤器我们先不分析，直接来到FailoverClusterInvoker。
1.1 FailoverClusterInvoker.
FailoverClusterInvoker 这个类是干什么的呢，可以从类名看出来，这个是一个失败重试的类。 先看下继承关系:
我们可以看到FailoverClusterInvoker 继承AbstractClusterInvoker，AbstractClusterInvoker这个抽象类其实还有很多子类，我们可以看到这些都是，
然后这个invoke方法是在父类里面的，然后由子类实现doInvoke方法。我们先来看下这个AbstractClusterInvoker 的invoke方法:
public Result invoke(final Invocation invocation) throws RpcException {
checkWhetherDestroyed();
// binding attachments into invocation.
// Map<String, Object> contextAttachments = RpcContext.getClientAttachment().getObjectAttachments();
// if (contextAttachments != null && contextAttachments.size() != 0) {
// ((RpcInvocation) invocation).addObjectAttachmentsIfAbsent(contextAttachments);
// }
// 通过路由策略，将不符合路由规则的invoker过滤掉，获取所有提供者的集合
List<Invoker<T>> invokers = list(invocation);
// 获取负载均衡策略，并创建相应的负载均衡实例,默认是random
LoadBalance loadbalance = initLoadBalance(invokers, invocation);
RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation);
// 调用具体的集群容错策略中的doInvoke()
return doInvoke(invocation, invokers, loadbalance);
}
- 首先检查是否销毁
- 将上下文中的附加信息添加到invocation 中。
- 获取invoker集合，获取负载均衡策略（这个后面讲）
- 调用子类的doInvoke(invocation, invokers, loadbalance) 方法。
我们来看下 FailoverClusterInvoker 的doInvoke 方法。
public Result doInvoke(Invocation invocation, final List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException {
List<Invoker<T>> copyInvokers = invokers;
// 检测invokers列表是否为空
checkInvokers(copyInvokers, invocation);
// 获取RPC调用的方法名
String methodName = RpcUtils.getMethodName(invocation);
// 获取retries属性值
int len = calculateInvokeTimes(methodName);
// retry loop.
RpcException le = null; // last exception.
// 存放所有已经尝试调用过的invoker，这些invoker中，除了最后一个外，其它的都是不可用的
List<Invoker<T>> invoked = new ArrayList<Invoker<T>>(copyInvokers.size()); // invoked invokers.
Set<String> providers = new HashSet<String>(len);
for (int i = 0; i < len; i++) {
//Reselect before retry to avoid a change of candidate `invokers`.
//NOTE: if `invokers` changed, then `invoked` also lose accuracy.
if (i > 0) {
// 检测委托对象invoker是否被销毁
checkWhetherDestroyed();
// 更新本地invoker列表
copyInvokers = list(invocation);
// check again 重新检测invokers列表是否为空
checkInvokers(copyInvokers, invocation);
}
// 负载均衡
Invoker<T> invoker = select(loadbalance, invocation, copyInvokers, invoked);
// 将选择出的invoker写入到invoked集合
invoked.add(invoker);
RpcContext.getServiceContext().setInvokers((List) invoked);
try {
// 远程调用
Result result = invokeWithContext(invoker, invocation);
//重试过程中，将最后一次调用的异常信息以 warn 级别日志输出
if (le != null && logger.isWarnEnabled()) {
logger.warn("Although retry the method " + methodName
+ " in the service " + getInterface().getName()
+ " was successful by the provider " + invoker.getUrl().getAddress()
+ ", but there have been failed providers " + providers
+ " (" + providers.size() + "/" + copyInvokers.size()
+ ") from the registry " + directory.getUrl().getAddress()
+ " on the consumer " + NetUtils.getLocalHost()
+ " using the dubbo version " + Version.getVersion() + ". Last error is: "
+ le.getMessage(), le);
}
return result;
} catch (RpcException e) {
// 如果是业务性质的异常，不再重试，直接抛出
if (e.isBiz()) { // biz exception.
throw e;
}
// 其他性质的异常统一封装成RpcException
le = e;
} catch (Throwable e) {
le = new RpcException(e.getMessage(), e);
} finally {
// 将提供者的地址添加到providers
providers.add(invoker.getUrl().getAddress());
}
} // end-for
// 最后抛出异常
throw new RpcException(le.getCode(), "Failed to invoke the method "
+ methodName + " in the service " + getInterface().getName()
+ ". Tried " + len + " times of the providers " + providers
+ " (" + providers.size() + "/" + copyInvokers.size()
+ ") from the registry " + directory.getUrl().getAddress()
+ " on the consumer " + NetUtils.getLocalHost() + " using the dubbo version "
+ Version.getVersion() + ". Last error is: "
+ le.getMessage(), le.getCause() != null ? le.getCause() : le);
}
其他我们不分析，直接进图DubboInvoker.doInvoker():
protected Result doInvoke(final Invocation invocation) throws Throwable {
RpcInvocation inv = (RpcInvocation) invocation;
final String methodName = RpcUtils.getMethodName(invocation);
inv.setAttachment(PATH_KEY, getUrl().getPath());
inv.setAttachment(VERSION_KEY, version);
ExchangeClient currentClient; // 一个client连接着一个Netty Client
if (clients.length == 1) { // 若只有一个client，则直接选择
currentClient = clients[0];
} else { // 若有多个client，则轮询选择一个
currentClient = clients[index.getAndIncrement() % clients.length];
}
try { // 若只发请求，无需server给出响应，则为oneway，否则为twoway
boolean isOneway = RpcUtils.isOneway(getUrl(), invocation);
int timeout = calculateTimeout(invocation, methodName);
invocation.put(TIMEOUT_KEY, timeout);
if (isOneway) {
boolean isSent = getUrl().getMethodParameter(methodName, Constants.SENT_KEY, false);
currentClient.send(inv, isSent);
return AsyncRpcResult.newDefaultAsyncResult(invocation);
} else {
ExecutorService executor = getCallbackExecutor(getUrl(), inv);
// todo
CompletableFuture<AppResponse> appResponseFuture = // 通过client提交请求
currentClient.request(inv, timeout, executor).thenApply(obj -> (AppResponse) obj);
// save for 2.6.x compatibility, for example, TraceFilter in Zipkin uses com.alibaba.xxx.FutureAdapter
FutureContext.getContext().setCompatibleFuture(appResponseFuture);
AsyncRpcResult result = new AsyncRpcResult(appResponseFuture, inv);
result.setExecutor(executor);
return result;
}
} catch (TimeoutException e) {
throw new RpcException(RpcException.TIMEOUT_EXCEPTION, "Invoke remote method timeout. method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
} catch (RemotingException e) {
throw new RpcException(RpcException.NETWORK_EXCEPTION, "Failed to invoke remote method: " + invocation.getMethodName() + ", provider: " + getUrl() + ", cause: " + e.getMessage(), e);
}
}
接下来直接到Nettyhannel.send()中，
@Override
public void send(Object message, boolean sent) throws RemotingException {
// whether the channel is closed
super.send(message, sent);
boolean success = true;
int timeout = 0;
try {
// todo
ChannelFuture future = channel.writeAndFlush(message); // 终于找到了
if (sent) {
// wait timeout ms
timeout = getUrl().getPositiveParameter(TIMEOUT_KEY, DEFAULT_TIMEOUT);
success = future.await(timeout);
}
Throwable cause = future.cause();
if (cause != null) {
throw cause;
}
} catch (Throwable e) {
removeChannelIfDisconnected(channel);
throw new RemotingException(this, "Failed to send message " + PayloadDropper.getRequestWithoutData(message) + " to " + getRemoteAddress() + ", cause: " + e.getMessage(), e);
}
if (!success) {
throw new RemotingException(this, "Failed to send message " + PayloadDropper.getRequestWithoutData(message) + " to " + getRemoteAddress()
+ "in timeout(" + timeout + "ms) limit");
}
}
在这里channel.writeAndFlush(message)将发送的请求发送出去了。接下来就是Netty的pipeline链处理了：
2. 提供者处理消费者请求
首先，看一下NettyServer的调用链：
ch.pipeline()
.addLast("decoder", adapter.getDecoder())
.addLast("encoder", adapter.getEncoder())
.addLast("server-idle-handler", new IdleStateHandler(0, 0, idleTimeout, MILLISECONDS))
.addLast("handler", nettyServerHandler);
直接进入nettyServerHadnler:
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
NettyChannel channel = NettyChannel.getOrAddChannel(ctx.channel(), url, handler);
handler.received(channel, msg);
}
接下来到AllChannelHandler：
// 当前类称为请求分发器Dispatcher
@Override
public void received(Channel channel, Object message) throws RemotingException {
// 线程池
ExecutorService executor = getPreferredExecutorService(message);
try {
// 将对端请求/响应封装为一个任务
// 从线程池中拿到一个线程，来处理这个任务。
executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
} catch (Throwable t) {
if(message instanceof Request && t instanceof RejectedExecutionException){
sendFeedback(channel, (Request) message, t);
return;
}
throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
}
}
ChannelEventRunnable.run():
public void run() {
if (state == ChannelState.RECEIVED) {
try {
handler.received(channel, message);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is " + message, e);
}
} else {
switch (state) {
case CONNECTED:
try {
handler.connected(channel);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
}
break;
case DISCONNECTED:
try {
handler.disconnected(channel);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel, e);
}
break;
case SENT:
try {
handler.sent(channel, message);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is " + message, e);
}
break;
case CAUGHT:
try {
handler.caught(channel, exception);
} catch (Exception e) {
logger.warn("ChannelEventRunnable handle " + state + " operation error, channel is " + channel
+ ", message is: " + message + ", exception is " + exception, e);
}
break;
default:
logger.warn("unknown state: " + state + ", message is " + message);
}
}
}
org.apache.dubbo.remoting.exchange.support.header.HeaderExchangeHandler#received:
public void received(Channel channel, Object message) throws RemotingException {
final ExchangeChannel exchangeChannel = HeaderExchangeChannel.getOrAddChannel(channel);
if (message instanceof Request) { // 处理client请求的情况
// handle request.
Request request = (Request) message;
if (request.isEvent()) {
handlerEvent(channel, request);
} else {
if (request.isTwoWay()) { // 处理双向请求
handleRequest(exchangeChannel, request);
} else {
handler.received(exchangeChannel, request.getData());
}
}
} else if (message instanceof Response) { // 处理server响应的情况
handleResponse(channel, (Response) message);
} else if (message instanceof String) {
if (isClientSide(channel)) {
Exception e = new Exception("Dubbo client can not supported string message: " + message + " in channel: " + channel + ", url: " + channel.getUrl());
logger.error(e.getMessage(), e);
} else {
String echo = handler.telnet(channel, (String) message);
if (echo != null && echo.length() > 0) {
channel.send(echo);
}
}
} else {
handler.received(exchangeChannel, message);
}
}
void handleRequest(final ExchangeChannel channel, Request req) throws RemotingException {
Response res = new Response(req.getId(), req.getVersion());
if (req.isBroken()) { // 判断请求是否已经发生过中断(异常)
Object data = req.getData();
String msg;
if (data == null) {
msg = null;
} else if (data instanceof Throwable) {
msg = StringUtils.toString((Throwable) data);
} else {
msg = data.toString();
}
res.setErrorMessage("Fail to decode request due to: " + msg);
res.setStatus(Response.BAD_REQUEST);
channel.send(res);
return;
}
// find handler by message class.
Object msg = req.getData();
try {
CompletionStage<Object> future = handler.reply(channel, msg); // 处理调用
future.whenComplete((appResult, t) -> { // 添加监听，一旦异步操作完成，就会触发该回调
try {
if (t == null) {
res.setStatus(Response.OK);
res.setResult(appResult);
} else {
res.setStatus(Response.SERVICE_ERROR);
res.setErrorMessage(StringUtils.toString(t));
}
channel.send(res);
} catch (RemotingException e) {
logger.warn("Send result to consumer failed, channel is " + channel + ", msg is " + e);
}
});
}
}
接下来DubboProtocol.reply():
private ExchangeHandler requestHandler = new ExchangeHandlerAdapter() {
@Override
public CompletableFuture<Object> reply(ExchangeChannel channel, Object message) throws RemotingException {
if (!(message instanceof Invocation)) {
throw new RemotingException(channel, "Unsupported request: "
+ (message == null ? null : (message.getClass().getName() + ": " + message))
+ ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress());
}
Invocation inv = (Invocation) message;
// todo
Invoker<?> invoker = getInvoker(channel, inv); // 获取invoker
...
RpcContext.getServiceContext().setRemoteAddress(channel.getRemoteAddress());
Result result = invoker.invoke(inv); // 完成invoker的本地调用计算
return result.thenApply(Function.identity()); // 将result构建为一个异步结果
}
}
getInvoker（）：
Invoker<?> getInvoker(Channel channel, Invocation inv) throws RemotingException {
boolean isCallBackServiceInvoke = false;
boolean isStubServiceInvoke = false;
int port = channel.getLocalAddress().getPort();
String path = (String) inv.getObjectAttachments().get(PATH_KEY);
// if it's callback service on client side
isStubServiceInvoke = Boolean.TRUE.toString().equals(inv.getObjectAttachments().get(STUB_EVENT_KEY));
if (isStubServiceInvoke) {
port = channel.getRemoteAddress().getPort();
}
//callback
isCallBackServiceInvoke = isClientSide(channel) && !isStubServiceInvoke;
if (isCallBackServiceInvoke) {
path += "." + inv.getObjectAttachments().get(CALLBACK_SERVICE_KEY);
inv.getObjectAttachments().put(IS_CALLBACK_SERVICE_INVOKE, Boolean.TRUE.toString());
}
String serviceKey = serviceKey(
port,
path,
(String) inv.getObjectAttachments().get(VERSION_KEY),
(String) inv.getObjectAttachments().get(GROUP_KEY)
);
// 从缓存map中获取exporter
DubboExporter<?> exporter = (DubboExporter<?>) exporterMap.get(serviceKey);
if (exporter == null) {
throw new RemotingException(channel, "Not found exported service: " + serviceKey + " in " + exporterMap.keySet() + ", may be version or group mismatch " +
", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress() + ", message:" + getInvocationWithoutData(inv));
}
return exporter.getInvoker(); // 获取exporter中封装的invoker
}
整体流程图：
红色箭头自下往上执行，首先是接到请求，进行decode解码操作，然后调用handlers的recevied方法，直到调用到实现类里面。蓝色的时候将响应写回到channel中，进行encode，然后调用handler的sent方法。
2.1 调用图
3. 消费者处理提供者响应
整体调用图：
3.1 NettyClientHandler
这里这个handler其实是代表的handlers， 因为我们在上篇文章invoke调用的时候，可以发现一个requestHandler被包装了若干层，我们可以再来回顾下那个netty将消息发送出去，然后调用handler的sent方法的图：
我们可以看到这个requestHandler被包装了这么多层。其实我们这里接收消息也差不多，只不过不是调用sent方法了，而是recevied方法。我们可以看下我画的这个handlers处理recevied的一个图：
可以看到与上面那张图差不多，这里我对每个handler都做了什么事做了一下总结（这里的总结只是对响应的处理，其他处理还需要自己犯翻下源码），这里还是有2个注意的handler我们需要看下 一个是AllChannelHandler，我们可以看下这个它的received源码：
这里是现获取线程池，然后使用线程池来处理这个事件。
出现了异常，如果message是request，然后是个线程池拒绝异常，这时候快速回复一个失败的响应。
我们来看下这个ChannelEventRunnable ，看看run方法是怎样处理这些事件的。
我们可以看到，不同的事件交给handler的不同方法来执行。
另一个是HeaderExchangeHandler，我们看下它的received源码
可以看出来，使用message的类型来处理，我们这里就看下这个message是response类型的时候那个handleResponse方法。\
// 处理响应
static void handleResponse(Channel channel, Response response) throws RemotingException {
// 不为null,也不是心跳
if (response != null && !response.isHeartbeat()) {
DefaultFuture.received(channel, response);
}
}
接下来DefaultFuture.received(）：
public static void received(Channel channel, Response response) {
received(channel, response, false);
}
public static void received(Channel channel, Response response, boolean timeout) {
try {
// 从futures 缓存中将 这个请求id 删除
DefaultFuture future = FUTURES.remove(response.getId());
if (future != null) {
Timeout t = future.timeoutCheckTask;
if (!timeout) {
// decrease Time
t.cancel();
}
// 接收响应
future.doReceived(response);
} else {
logger.warn("The timeout response finally returned at "
+ (new SimpleDateFormat("yyyy-MM-dd HH:mm:ss.SSS").format(new Date()))
+ ", response status is " + response.getStatus()
+ (channel == null ? "" : ", channel: " + channel.getLocalAddress()
+ " -> " + channel.getRemoteAddress()) + ", please check provider side for detailed result.");
}
} finally {
// 最终将channels 缓存中将请求id删除
CHANNELS.remove(response.getId());
}
}
这里其实就是找到请求那时候的那个future，这里response.getId()获得的id其实就是当初request那个id。 如果future能在缓存中找到，就调用future对象的doReceived方法。找不到的话，就是超时了，然后被清理了。 我们接着在看future.doReceived(response)方法
private void doReceived(Response res) {
if (res == null) {
throw new IllegalStateException("response cannot be null");
}
if (res.getStatus() == Response.OK) {
this.complete(res.getResult());
} else if (res.getStatus() == Response.CLIENT_TIMEOUT || res.getStatus() == Response.SERVER_TIMEOUT) {
this.completeExceptionally(new TimeoutException(res.getStatus() == Response.SERVER_TIMEOUT, channel, res.getErrorMessage()));
} else {
this.completeExceptionally(new RemotingException(channel, res.getErrorMessage()));
}
// the result is returning, but the caller thread may still waiting
// to avoid endless waiting for whatever reason, notify caller thread to return.
if (executor != null && executor instanceof ThreadlessExecutor) {
ThreadlessExecutor threadlessExecutor = (ThreadlessExecutor) executor;
if (threadlessExecutor.isWaiting()) {
threadlessExecutor.notifyReturn(new IllegalStateException("The result has returned, but the biz thread is still waiting" +
" which is not an expected state, interrupt the thread manually by returning an exception."));
}
}
}
3.2 整体调用图
接下里，我们将分析服务路由，敬请期待。
