10、Dubbo源码系列-几种线程模型

YMCODE
106 阅读6分钟

“我报名参加金石计划1期挑战——瓜分10万奖池，这是我的第4篇文章，点击查看活动详情”

旧书不厌百回读，熟读精思子自知。周末看了看中国说唱巅峰对决，和朋友互相吐槽了各自的业务，去新房看了下装修进度，回到家尚有精力，正好趁着这次金石计划，就继续和大家聊下Dubbo的线程模型吧

一、概述

想必大家都知道，Dubbo底层网络通信使用的Netty，默认其实是两级线程模型，分别为Boss线程、worker线程。 Boss线程主要用来处理 TCP的链接请求，wokrer线程用来处理read or write事件。 当服务提供方逻辑简单，接口耗时比较低的的嘶吼，直接用I/O线程处理会更快，减少多线程CPU上下文切换；但是如果服务端逻辑处理比较耗时，如果还占用I/O线程的话，则会阻塞I/O线程，使得Netty不能很好的处理客户端请求。基于此Dubbo其实提供了多种线程模型实现。

  • AllDispater
    • 所有的消息都派发给业务线程，如请求、响应、连接事件、断开事件、心跳事件等等。
  • DirectDispatcher
    • 所有的消息都不派发给业务线程，全部在I/O线程上执行。
  • MessageOnlyDispatcher
    • 只有请求响应消息派发给业务线程池，其他消息都在I/O线程上执行。
  • ExecutionDispatcher
    • 只把请求类消息发给业务线程，其他事件在I/O线程上执行。
  • ConnectionOrderedDispatcher
    • I/O线程用来处理连接、断开事件，其他消息业务线程处理。

二、线程模型初始化

不知道大家还记不记得前文讲服务端初始化流程时，提到过NettyServer的初始化，代码如下：

    public NettyServer(URL url, ChannelHandler handler) throws RemotingException {
        super(ExecutorUtil.setThreadName(url, SERVER_THREAD_POOL_NAME), ChannelHandlers.wrap(handler, url));
    }

这里关注下ChannelHandlers.wrap(handler, url)方法，内部调用wrapInternal来实现对ChannelHandlers的包装，可以看到，最终根据URL里的线程模型配置，来选择对应的实现类，调用其dispatch方法。

    protected ChannelHandler wrapInternal(ChannelHandler handler, URL url) {
        ChannelHandler wrapperHanlder = ExtensionLoader.getExtensionLoader(Dispatcher.class)
                .getAdaptiveExtension().dispatch(handler, url);
        return new MultiMessageHandler(new HeartbeatHandler(wrapperHanlder));
    }

查看Dispatcher源码如下：

@SPI(AllDispatcher.NAME)
public interface Dispatcher {
    @Adaptive({Constants.DISPATCHER_KEY, "dispather", "channel.handler"})
    // The last two parameters are reserved for compatibility with the old configuration
    ChannelHandler dispatch(ChannelHandler handler, URL url);
}

可以看到又是一个SPI注入点，默认实现为AllDispatcher，同时提供了多种实现，下面就让我们来一一查看各自的实现吧。 image.png

三、原理剖析

3.1. AllDispater

  • 线程模型示意图 image.png
  • AllDispater源码
public class AllDispatcher implements Dispatcher {
    public static final String NAME = "all";
    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new AllChannelHandler(handler, url);
    }
}

通过上面得知，dispatch方法返回了AllChannelHandler对象，查看AllChannelHandler实现如下：

public class AllChannelHandler extends WrappedChannelHandler {

    public AllChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
    }

    @Override
    public void connected(Channel channel) throws RemotingException {
        ExecutorService executor = getExecutorService();
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CONNECTED));
    }

    @Override
    public void disconnected(Channel channel) throws RemotingException {
        ExecutorService executor = getExecutorService();
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.DISCONNECTED));
    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
        ExecutorService executor = getPreferredExecutorService(message);
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
        
    }

    @Override
    public void caught(Channel channel, Throwable exception) throws RemotingException {
        ExecutorService executor = getExecutorService();
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CAUGHT, exception));

    }
}

首先介绍下WrappedChannelHandler类，默认使用I/O线程处理所有事件，子类会对其方法进行覆盖，从而使用业务线程池进行事件处理。 通过上述代码可以看到，connected、disconnected、received、caught均使用dubbo线程池，sent不使用dubbo线程池，直接使用io线程池。

3.2. DirectDispatcher

  • 线程模型示意图 image.png
  • DirectDispatcher源码
public class DirectDispatcher implements Dispatcher {
    public static final String NAME = "direct";
    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new DirectChannelHandler(handler, url);
    }

}

通过上面得知，dispatch方法返回了DirectChannelHandler对象，查看DirectChannelHandler实现如下：

public class DirectChannelHandler extends WrappedChannelHandler {

    public DirectChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
        ExecutorService executor = getPreferredExecutorService(message);
        if (executor instanceof ThreadlessExecutor) {
            try {
                executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
            } catch (Throwable t) {
                throw new ExecutionException(message, channel, getClass() + " error when process received event .", t);
            }
        } else {
            handler.received(channel, message);
        }
    }

}

DirectDispatcher类重写了received方法，但还会在当前IO线程执行执行，其他事件connected、disconnected、caught、sent均不使用dubbo线程池，直接使用io线程池。

3.3. MessageOnlyDispatcher

  • 线程模型示意图 image.png
  • MessageOnlyDispatcher源码
public class MessageOnlyDispatcher implements Dispatcher {
    public static final String NAME = "message";
    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new MessageOnlyChannelHandler(handler, url);
    }
}

通过上面得知，dispatch方法返回了MessageOnlyChannelHandler对象，查看MessageOnlyChannelHandler实现如下：

public class MessageOnlyChannelHandler extends WrappedChannelHandler {

    public MessageOnlyChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
        ExecutorService executor = getPreferredExecutorService(message);
        executor.execute(new ChannelEventRunnable(channel, handler,
    }

}

查看源码可得知，received时间使用dubbo线程池，connected、disconnected、sent、caught不使用dubbo线程池，直接使用io线程池。

3.4. ExecutionDispatcher

  • 线程模型示意图 image.png
  • ExecutionDispatcher源码
public class ExecutionDispatcher implements Dispatcher {
    public static final String NAME = "execution";
    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new ExecutionChannelHandler(handler, url);
    }
}

通过上面得知，dispatch方法返回了ExecutionChannelHandler对象，查看ExecutionChannelHandler实现如下：

public class ExecutionChannelHandler extends WrappedChannelHandler {

    public ExecutionChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
        ExecutorService executor = getPreferredExecutorService(message);
        // 只把请求类消息发给业务线程处理
        if (message instanceof Request) {
            try {
                executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
            } catch (Throwable t) {
                if (t instanceof RejectedExecutionException) {
                    sendFeedback(channel, (Request) message, t);
                }
                throw new ExecutionException(message, channel, getClass() + " error when process received event.", t);
            }
        } else if (executor instanceof ThreadlessExecutor) {
            executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
        } else {
            // 其他事件，则直接在I/O线程上执行
            handler.received(channel, message);
        }
    }
}

查看源码可以发现，只有请求类的消息交给了业务线程处理，其他事件都是I/O线程进行处理。

3.5. ConnectionOrderedDispatcher

  • 线程模型示意图 image.png
  • DirectDispatcher源码
public class ConnectionOrderedDispatcher implements Dispatcher {
    public static final String NAME = "connection";
    @Override
    public ChannelHandler dispatch(ChannelHandler handler, URL url) {
        return new ConnectionOrderedChannelHandler(handler, url);
    }
}

通过上面得知，dispatch方法返回了ConnectionOrderedChannelHandler对象，查看ConnectionOrderedChannelHandler实现如下：

    public ConnectionOrderedChannelHandler(ChannelHandler handler, URL url) {
        super(handler, url);
        String threadName = url.getParameter(THREAD_NAME_KEY, DEFAULT_THREAD_NAME);
        connectionExecutor = new ThreadPoolExecutor(1, 1,
                0L, TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<Runnable>(url.getPositiveParameter(CONNECT_QUEUE_CAPACITY, Integer.MAX_VALUE)),
                new NamedThreadFactory(threadName, true),
                new AbortPolicyWithReport(threadName, url)
        );  // FIXME There's no place to release connectionExecutor!
        queuewarninglimit = url.getParameter(CONNECT_QUEUE_WARNING_SIZE, DEFAULT_CONNECT_QUEUE_WARNING_SIZE);
    }
    
  @Override
    public void connected(Channel channel) throws RemotingException {
         connectionExecutor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CONNECTED));
    }

    @Override
    public void disconnected(Channel channel) throws RemotingException {
         connectionExecutor.execute(new ChannelEventRunnable(channel, handler, ChannelState.DISCONNECTED));

    }

    @Override
    public void received(Channel channel, Object message) throws RemotingException {
        ExecutorService executor = getPreferredExecutorService(message);
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.RECEIVED, message));
      

    @Override
    public void caught(Channel channel, Throwable exception) throws RemotingException {
        ExecutorService executor = getExecutorService();
        executor.execute(new ChannelEventRunnable(channel, handler, ChannelState.CAUGHT, exception));
   
    }

可以看到，构造函数里创建了一个直包含一个线程的线程池和一个有限元素的队列，建立链接、断开链接事件放入线程池中，单线程进行处理，把建立链接、断开链接进行顺序处理。请求事件和异常事件则是交给了业务线程处理。其他事件都是I/O事件进行处理。

四、小节

本篇文章主要为大家介绍了下Dubbo的线程模型选择时机以及各自的实现原理，可以看到，线程模型的注入也是SPI实现的，如果你觉得以上线程模型都不适合你的业务场景，同样可以通过自己实现Dispatcher接口，添加自己的xxxChannelHandler实现，最后在自己项目的resource/META-INFO下创建对应的声明文件即可。

avatar
文章
阅读
粉丝