Netty原理和源码个人分析

49 阅读11分钟

大致走了一遍Netty启动过程,分析了一波加深印象,也学会了NIO设计思路,希望能给各位提供一些帮助,也希望大家能多多指教,后面还要更加深入学习Netty这个框架,内容很多

Netty源码分析

一、启动时所有方法依次分析:

1、new NioEventLoopGroup():

NioEventLoopGroup --> MultithreadEventLoopGroup --> MultithreadEventExecutorGroup

每个类重点

MultithreadEventLoopGroup中确定了EventLoopGroup中线程数是多少,默认是CPU * 2
MultithreadEventExecutorGroup中创建了Children数组,里面是NioEventLoop,然后对Children初始化(创建NioEventLoop)
	

2、new NioEventLoop():

NioEventLoop --> SingleThreadEventLoop --> SingleThreadEventExecutor
		
每个类重点
NioEventLoop中创建并open了Selector,并且将Selector中的SeletionKey由原来的HashSet换成了Netty自己实现的数组Set
SingleThreadEventExecutor中有一个taskQueue用来存放此线程的任务,还有一个Thread/Executor用来保存当前EventLoop的线程
		

2.5、new NioSocketChannel():

NioSocketChannel --> AbstractNioByteChannel --> AbstractNioChannel

每个类重点
NioSocketChannel中创建并open了java原生的socketchannel
AbstractNioChannel中将java原生的socketchannel设置为非阻塞模式,并且设置了类变量readInterestOp为SelectionKey.OP_READ
AbstractChannel中给每个channel分配了唯一id,创建了一个Unsafe实例以及创建了一个新的Pipeline(DefaultChannelPipeline)也就是说,创建了NioSocketChannel后,类中已经将NioSocketChannel和原生socketchannel相关联
		

3、boostrap.channel(NioSocketChannel.class) /boostrap.channel(NioServerSocketChannel.class)

往boostrap中的channelFactory注册对应SocketChannel的ReflectiveChannelFactory,ReflectiveChannelFactory中的newChannel会返回对应的实例对象
	

4、connect()

connect() --> doResolveAndConnect()
		
private ChannelFuture doResolveAndConnect(final SocketAddress remoteAddress, final SocketAddress localAddress) {
        ①final ChannelFuture regFuture = initAndRegister();
        final Channel channel = regFuture.channel();

        if (regFuture.isDone()) {
                if (!regFuture.isSuccess()) {
                        return regFuture;
                }
                return ②doResolveAndConnect0(channel, remoteAddress, localAddress, channel.newPromise());
        } else {
                ......
        }
}
		

doResolveAndConnect()有两个重要方法

(1).initAndRegister()中重要方法

    channel = channelFactory.newChannel() 

此方法调用了上面注册的Factory.newChannel, 因此返回了一个NioSocketChannel

    init(channel)

此方法调用了boostrap中的init方法,将在创建boostrap过程中在handler中设置的ChannelInitializer注册到channel上,此channel 为上一步新建的NioSocketChannel,此时channel中pipeline为 : head --- ChannelInitializer --- tail 然后设置channel为创建boostrap过程中设置的Option属性。

    config().group().register(channel)

group为上面新建的NioEventLoopGroup,register方法从Group中选择一个NioEventLoop执行register(),最后会执行 channel.unsafe().register(this, promise); 这里的channel是上面新建的channel。在这个方法中,他会将调用register方法的 EventLoop赋值给当前channel(新建的NioSocketChannel)的EventLoop属性,完成socketchannel和eventloop的绑定。

具体方法为: AbstractChannel.this.eventLoop = eventLoop; 然后会判断此线程是否是eventLoop线程,如果不是则将注册方法register0(promise)加入到eventLoop任务队列中,因此接下来的 register0方法会在eventLoop线程中运行,发生线程切换,主线程在此处返回promise,并调用promise.cause()阻塞等待结果。

    register0() --> AbstractChannel方法

执行此方法的线程的eventLoop线程,首先执行doRegister()-->AbstractNioChannel方法,此方法执行selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);即将java原生socketchannel注册到selector上(此selector来自当前eventLoop线程,在上面注册过程中已经open,并且channel上已经绑定了eventLoop,所以可以从channel中获取eventLoop,进而获取selector。

此处register的attachment参数为channel本身,于是便将java原生nio socketchannel和NioSocketChannel在Selector上关联上。

然后执行pipeline.invokeHandlerAddedIfNeeded(), 此方法会将上面注册的ChannelInitializer中添加的handler全部添加到pipeline中,并将ChannelInitializer移除。

然后执行safeSetSuccess(promise),告知主线程注册方法已经结束,可以结束阻塞拿到结果。 然后执行pipeline.fireChannelRegistered(),凡是fireChannelXXX()方法都是往channel的pipeline中激活对应事件的handler执行对应方法,像此处的fireChannelRegistered(),从head到tail每一个inboundHandler都执行channelRegistered()方法。然后执行pipeline.fireChannelActive(),原理同上

总结:initAndRegister()将java原生的niochannel注册到selector上,并对pipeline初始化

经过上面所有操作,initAndRegister()方法已经完成,此时主线程已经结束等待并获得成功结果拿到创建的channel对象,并准备好执行接下来的方法了

(2).doResolveAndConnect0():

此方法中会真正调用java底层的socketchannel的connect方法

1.doConnect()方法:
private static void doConnect(
                final SocketAddress remoteAddress, final SocketAddress localAddress, final ChannelPromise connectPromise) {

        // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up
        // the pipeline in its channelRegistered() implementation.
        final Channel channel = connectPromise.channel();
        channel.eventLoop().execute(new Runnable() {
                @Override
                public void run() {
                        if (localAddress == null) {
                                channel.connect(remoteAddress, connectPromise);
                        } else {
                                channel.connect(remoteAddress, localAddress, connectPromise);
                        }
                        connectPromise.addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
                }
        });
}
				

此方法中的channelPromise是由上面新建的NioSocketChannel调用newPromise()创建,因此能拿到当前新建的channel,并从channel中获得此channel绑定的eventLoop,并且在eventLoop中添加了新的任务,此时再次线程切换来执行channel.connect()。

channel.connect()方法会在pipeline中所有handler依次执行connect方法,最后会在head(HeadContext)中执行connect方法,调用以下方法

@Override
public void connect(
                ChannelHandlerContext ctx,
                SocketAddress remoteAddress, SocketAddress localAddress,
                ChannelPromise promise) {
        unsafe.connect(remoteAddress, localAddress, promise);
}

在核心代码unsafe的connect方法中,会执行doConnect(remoteAddress, localAddress)方法,最后回到NioSocketChannel 执行doConnect()方法,此方法会执行

boolean connected = SocketUtils.connect(javaChannel(), remoteAddress);

在此处完成原生nio socketchannel的connect方法,完成后把promise设置成功并返回

二、ChannelPipeline分析

每个Channel都会绑定一个ChannelPipeline,同时pipeline中也会绑定对应的channel。在上面已经说过创建Channel过程中会同时创建一个ChannelPipeline。因此从channel中可以获得pipeline,从pipeline中也可以获得channel

ChannelHandler 是消息的具体处理器,主要负责处理客户端/服务端接收和发送的数据。
ChannelPipeline 则是包含了一个或多个 ChannelHandler 的链表。
有关ChannelPipeline的类分析
	
ChannelHandler --> ChannelInboundHandler/ChannelOutboundHandler --> ChannelInboundHandlerAdapter/ChannelOutboundHandlerAdapter

ChannelHandlerContext --> AbstractChannelHandlerContext(Pipeline中实际存在的每个结点)
	

ChannelHandlerContext和ChannelHandler的关系,可以从DefaultChannelHandlerContext的构造方法中得到:

DefaultChannelHandlerContext(
                DefaultChannelPipeline pipeline, EventExecutor executor, String name, ChannelHandler handler) {
        super(pipeline, executor, name, handler.getClass());
        this.handler = handler;
}

因此,可以认为ChannelHandlerContext是对ChannelHandler做了一层封装。再看AbstractChannelHandlerContext构造方法:

AbstractChannelHandlerContext(DefaultChannelPipeline pipeline, EventExecutor executor,
                  String name, Class<? extends ChannelHandler> handlerClass) {
        this.name = ObjectUtil.checkNotNull(name, "name");
        this.pipeline = pipeline;
        this.executor = executor;
        this.executionMask = mask(handlerClass);
        // Its ordered if its driven by the EventLoop or the given Executor is an instanceof OrderedEventExecutor.
        ordered = executor == null || executor instanceof OrderedEventExecutor;
}

因此每一个Context都会绑定一个executor(其实就是一个EventLoop),一个pipeline(通过此pipeline可以获得这个pipeline所在的channel,进而获得这个pipeline对应的EventLoop)

首先先从ChannelPipeline的addLast方法进行分析:

addLast(EventExecutorGroup group, ChannelHandler handler)

方法中,首先new了一个DefalutChannelHandlerContext(handler),同时从EventExecutorGroup(其实就是EventLoopGroup)中获取一个eventloop,并赋值到new出来的context的executor上, 完成EventLoopGroup和DefalutChannelHandlerContext的绑定,也就是可以选择不同的EventLoopGroup(中的一个eventloop)来执行对应的handler。最后将此ChannelHandlerContext加入到pipeline中

接下来研究DefaultChannelPipeline中的管道执行方法,选择研究fireChannelRead,其余fire方法同理(fireChannelXXX方法是使用pipeline处理对应动作的入口)

public final ChannelPipeline fireChannelRead(Object msg) {
        AbstractChannelHandlerContext.invokeChannelRead(head, msg);
        return this;
}

因此可以看出,fireChannelRead是先从头结点开始执行,msg为接收到的数据

查看AbstractChannelHandlerContext的类方法invokeChannelRead:

static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
        final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
        //executor方法会返回此channel对应的eventLoop,说明对于head来说,它的方法必定会在自己的evenloop内执行
        EventExecutor executor = next.executor();

        if (executor.inEventLoop()) {
                next.invokeChannelRead(m);
        } else {
                executor.execute(new Runnable() {
                        @Override
                        public void run() {
                                next.invokeChannelRead(m);
                        }
                });
        }
}

注:AbstractChannelHandlerContext对象的executor方法,由此可见如果没有给结点指定对应的eventloop,那么就会用channel自己的eventloop执行

public EventExecutor executor() {
        if (executor == null) {
                return channel().eventLoop();
        } else {
                return executor;
        }
}

查看AbstractChannelHandlerContext对象方法next.invokeChannelRead(m)方法

private void invokeChannelRead(Object msg) {
        if (invokeHandler()) {
                try {
                        ((ChannelInboundHandler) handler()).channelRead(this, msg);
                } catch (Throwable t) {
                        notifyHandlerException(t);
                }
        } else {
                fireChannelRead(msg);
        }
}

handler()方法会返回绑定在此DefalutChannelHandlerContext上的ChannelHandler,并调用其上的channelRead方法。选择ChannelInboundHandlerAdapter查看,其最后会调用

ctx.fireChannelRead(msg)

此处的ctx为head(ChannelHandlerContext)于是跟进去查看,此方法在AbstractChannelHandlerContext中实现,是对象方法,与Pipeline中的fireChannelRead方法不同

public ChannelHandlerContext fireChannelRead(final Object msg) {
        invokeChannelRead(findContextInbound(MASK_CHANNEL_READ), msg);
        return this;
}

又返回去调用类invokeChannelRead方法,findContextInbound方法是根据当前handler查找下一个对应handler的方法,因此又一次循环 进入上一个过程,直到尾结点tail的ChannelRead方法调用,由此分析完ChannelPipeline的过程。 (此处提示我们,如果想要这个msg传入下一个handler,需要调用ctx.firechannelRead)

三、Future、Promise分析

Future(Java原生) --> Future(netty) --> ChannelFuture --> ChannelPromise

Future(Java原生) --> Future(netty) --> Promise --> ChannelPromise

从上可知,无论是Future(netty)、还是Promise,都是在java原生Future的基础上增强的,以下分析DefaultPromise,其余类则基本一致。接下来从DefaultPromise的主要方法进行分析

1.sync()/await():

sync()方法里面直接调用了await(),因此分析await(),直接看代码:

@Override
public Promise<V> await() throws InterruptedException {
        if (isDone()) {
                return this;
        }

        if (Thread.interrupted()) {
                throw new InterruptedException(toString());
        }

        checkDeadLock();

        synchronized (this) {
                while (!isDone()) {
                        incWaiters();
                        try {
                                wait();
                        } finally {
                                decWaiters();
                        }
                }
        }
        return this;
}

可知,当调用了await方法,直接进入synchronized(锁住的对象为promise)后在调用Object类的wait()方法进入等待。

2.setSuccess(value)/setFailure(value) 方法

进入setValue0()方法
private boolean setValue0(Object objResult) {
        if (RESULT_UPDATER.compareAndSet(this, null, objResult) ||
                RESULT_UPDATER.compareAndSet(this, UNCANCELLABLE, objResult)) {
                if (checkNotifyWaiters()) {
                        notifyListeners();
                }
                return true;
        }
        return false;
}

在checkNotifyWaiters()(方法是被synchronized修饰的)中可以看见直接调用了notifyAll(),因此调用setSuccess(value)后会唤醒调用了await()/sync()方法的线程。接下来看notifyListeners()方法

private void notifyListeners() {
        EventExecutor executor = executor();
        if (executor.inEventLoop()) {
                final InternalThreadLocalMap threadLocals = InternalThreadLocalMap.get();
                final int stackDepth = threadLocals.futureListenerStackDepth();
                if (stackDepth < MAX_LISTENER_STACK_DEPTH) {
                        threadLocals.setFutureListenerStackDepth(stackDepth + 1);
                        try {
                                notifyListenersNow();
                        } finally {
                                threadLocals.setFutureListenerStackDepth(stackDepth);
                        }
                        return;
                }
        }

        safeExecute(executor, new Runnable() {
                @Override
                public void run() {
                        notifyListenersNow();
                }
        });
}

从中可以看出,Promise会在对应的EventLoop中调用notifyListenersNow()方法,跟进去后会调用以下方法

private void notifyListeners0(DefaultFutureListeners listeners) {
        GenericFutureListener<?>[] a = listeners.listeners();
        int size = listeners.size();
        for (int i = 0; i < size; i ++) {
                notifyListener0(this, a[i]);
        }
}
private static void notifyListener0(Future future, GenericFutureListener l) {
        try {
                l.operationComplete(future);
        } catch (Throwable t) {
                if (logger.isWarnEnabled()) {
                        logger.warn("An exception was thrown by " + l.getClass().getName() + ".operationComplete()", t);
                }
        }
}

因此,可以看到在调用setSuccess后,紧接着会在当前EventLoop中执行回调函数GenericFutureListener中的operationComplete方法。这个接口是由我们自己来实现的。至此,分析完Promise的异步过程,用一个例子结束:

public static void main(String[] args) {
        // 构造线程池
        EventExecutor executor = new DefaultEventExecutor();

        // 创建 DefaultPromise 实例
        Promise promise = new DefaultPromise(executor);

        // 下面给这个 promise 添加两个 listener
        promise.addListener(new GenericFutureListener<Future<Integer>>() {
                @Override
                public void operationComplete(Future future) throws Exception {
                        if (future.isSuccess()) {
                                System.out.println("任务结束,结果:" + future.get());
                        } else {
                                System.out.println("任务失败,异常:" + future.cause());
                        }
                }
        }).addListener(new GenericFutureListener<Future<Integer>>() {
                @Override
                public void operationComplete(Future future) throws Exception {
                        System.out.println("任务结束,balabala...");
                }
        });

        // 提交任务到线程池,五秒后执行结束,设置执行 promise 的结果
        executor.submit(new Runnable() {
                @Override
                public void run() {
                        try {
                                Thread.sleep(5000);
                        } catch (InterruptedException e) {
                        }
                        // 设置 promise 的结果
                        // promise.setFailure(new RuntimeException());
                        promise.setSuccess(123456);
                }
        });

        // main 线程阻塞等待执行结果
        try {
                promise.sync();
        } catch (InterruptedException e) {
        }
}
			
	
	

最后分析一下java原生Future的使用,加深印象。主要分析FutureTask,因为他是线程池中submit默认的Future实现

FutureTask <-- RunnableFuture <-- Future、Runnable

首先从构造方法分析:

public FutureTask(Runnable runnable, V result) {
        this.callable = Executors.callable(runnable, result);
        this.state = NEW;       // ensure visibility of callable
}
		

Callable的实现类RunnableAdapter在Excutors类中实现,构造方法如下

RunnableAdapter(Runnable task, T result) {
        this.task = task;
        this.result = result;
}

因此,Callable实际上是Runnable + result的组合,其中含有runnable任务,而FutureTask中的变量有Callable,因此也存在runnable任务

又因为FutureTask实现了Runnable(因为其实现了RunnableFuture),所以其里面含有run()方法,因此可以作为线程池execute(Runnable task)方法的参数。下面分析FutureTask的run方法:

public void run() {
        if (state != NEW ||
                !RUNNER.compareAndSet(this, null, Thread.currentThread()))
                return;
        try {
                Callable<V> c = callable;
                if (c != null && state == NEW) {
                        V result;
                        boolean ran;
                        try {
                                /**
                                        在RunnableAdapter中,call方法的实现为
                                        public T call() {
                                                task.run();
                                                return result;
                                        }
                                        因此在这里执行了在一开始submit(Runnable task, T result)传进来的 Runnable 方法,并把传进来的result返回
                                **/
                                result = c.call();
                                ran = true;
                        } catch (Throwable ex) {
                                result = null;
                                ran = false;
                                setException(ex);
                        }
                        if (ran)
                                set(result);
                }
        } finally {
                // runner must be non-null until state is settled to
                // prevent concurrent calls to run()
                runner = null;
                // state must be re-read after nulling runner to prevent
                // leaked interrupts
                int s = state;
                if (s >= INTERRUPTING)
                        handlePossibleCancellationInterrupt(s);
        }
}
		

接下来分析set(result)方法,set中设置了outcome属性为result, 把STATE状态CAS成了COMPLETING,并调用了finishCompletion(),此方法如下

private void finishCompletion() {
        // assert state > COMPLETING;
        for (WaitNode q; (q = waiters) != null;) {
                if (WAITERS.weakCompareAndSet(this, q, null)) {
                        for (;;) {
                                Thread t = q.thread;
                                if (t != null) {
                                        q.thread = null;
                                        LockSupport.unpark(t);
                                }
                                WaitNode next = q.next;
                                if (next == null)
                                        break;
                                q.next = null; // unlink to help gc
                                q = next;
                        }
                        break;
                }
        }

        done();

        callable = null;        // to reduce footprint
}

可以看出和AQS中同步队列十分相似,就是找到头结点并把后一个结点unpark掉,因此我们去看get()方法:

public V get() throws InterruptedException, ExecutionException {
        int s = state;
        if (s <= COMPLETING)
                s = awaitDone(false, 0L);
        return report(s);
}

get方法就是判断一下state状态是否是COMPLETING,由上可知只有调用了run方法后(也就是进入线程池执行Execute方法后)才会设置,因此会进入awaitDone方法,这部分代码十分长,但原理和AQS同步队列相似,都是用一个链表来存储线程结点,并调用LockSupport.park()让线程进入等待。

主要完成的事情如下:

private int awaitDone(boolean timed, long nanos)
                           

该方法用了一个死循环,首先判断此FutureTask是否已经完成(状态STATE是否为COMPLETING),如果是则直接Thread.yield(), 然后判断线程 是否被打断,如果打断了就从队列中移除并直接抛异常 如果都没有,则CAS到队尾中调用LockSupport.parkNanos(this, parkNanos);阻塞parkNanos后继续重复上面动作

四、NioEventLoopGroup/NioEventLoop 分析

NioEventLoopGroup <-- MultithreadEventLoopGroup <-- MultithreadEventExecutorGroup <-- AbstractEventExecutorGroup

首先分析NioEventLoopGroup,可以认为这个类就是一个线程池在NioEventLoopGroup的构造方法中,比较值得注意的部分有:

public NioEventLoopGroup(int nThreads, Executor executor, EventExecutorChooserFactory chooserFactory,
                                                 final SelectorProvider selectorProvider,
                                                 final SelectStrategyFactory selectStrategyFactory,
                                                 final RejectedExecutionHandler rejectedExecutionHandler,
                                                 final EventLoopTaskQueueFactory taskQueueFactory) {
        super(nThreads, executor, chooserFactory, selectorProvider, selectStrategyFactory,
                        rejectedExecutionHandler, taskQueueFactory);
}

executor默认值为空

selectorProvider默认值是由SelectorProvider.provider()生成的,即Java原生Selector

SelectStrategyFactory默认值是DefaultSelectStrategyFactory.INSTANCE

RejectedExecutionHandler默认值为RejectedExecutionHandlers.reject(),这个默认的拒绝策略是直接抛出异常

值得注意的是,以上三个参数都不是给EventLoopGroup构造时使用的,而是在接下来生成的EventLoop中使用的。接着分析MultithreadEventLoopGroup

protected MultithreadEventLoopGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
        super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, threadFactory, args);
}

比较重要的部分,设置了默认的线程池线程数,如果没有设置默认是CPU数*2。然后是MultithreadEventExecutorGroup构造方法

protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
                            EventExecutorChooserFactory chooserFactory, Object... args) {
        if (nThreads <= 0) {
                throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
        }

        if (executor == null) {
                executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
        }

        children = new EventExecutor[nThreads];

        for (int i = 0; i < nThreads; i ++) {
                boolean success = false;
                try {
                        children[i] = newChild(executor, args);
                        success = true;
                } catch (Exception e) {
                        // TODO: Think about if this is a good exception type
                        throw new IllegalStateException("failed to create a child event loop", e);
                } finally {
                        if (!success) {
                                for (int j = 0; j < i; j ++) {
                                        children[j].shutdownGracefully();
                                }

                                for (int j = 0; j < i; j ++) {
                                        EventExecutor e = children[j];
                                        try {
                                                while (!e.isTerminated()) {
                                                        e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
                                                }
                                        } catch (InterruptedException interrupted) {
                                                // Let the caller handle the interruption.
                                                Thread.currentThread().interrupt();
                                                break;
                                        }
                                }
                        }
                }
        }
}

Executor默认值为ThreadPerTaskExecutor

EventExecutorChooserFactory默认值为DefaultEventExecutorChooserFactory.INSTANCE

关键部分为:

children = new EventExecutor[nThreads];
children[i] = newChild(executor, args);

此方法由NioEventLoopGroup实现:

protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        EventLoopTaskQueueFactory queueFactory = args.length == 4 ? (EventLoopTaskQueueFactory) args[3] : null;
        return new NioEventLoop(this, executor, (SelectorProvider) args[0],
                ((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2], queueFactory);
}

由此可见,NioEventLoopGroup就是一个线程池,而每一个NioEventLoop就是线程池中的一个线程。接下来看NioEventLoop:

NioEventLoop <-- SingleThreadEventLoop <-- SingleThreadEventExecutor <-- AbstractScheduledEventExecutor <-- AbstractEventExecutor

接下来一个一个分析

1、NioEventLoop

selectorProvider为NioEventLoopGroup中的SelectorProvider.provider() SelectStrategy为NioEventLoopGroup中的DefaultSelectStrategyFactory.INSTANCE。

这里主要做了openSelector();这个方法,让每一个EventLoop自己开了一个selector。

2、SingleThreadEventExecutor

这里有一个属性TaskQueue,是每个提交到此线程的任务的存放位置,实现为LinkedBlockingQueue,最大容量为16。

executor属性为上面的ThreadPerTaskExecutor

rejectedExecutionHandler为NioEventLoopGroup中的RejectedExecutionHandlers.reject()

到此,NioEventLoop中的线程还没有启动,此时还只是一个类 分析execute方法,这个方法第一次被执行的时候是在initAndRegister()方法中,也就是每个NioSocketChannel执行register方法的初始化的时候第一次执行

public void execute(Runnable task) {
    if (task == null) {
            throw new NullPointerException("task");
    }

    boolean inEventLoop = inEventLoop();
    addTask(task);
    if (!inEventLoop) {
            startThread();
            if (isShutdown()) {
                    boolean reject = false;
                    try {
                            if (removeTask(task)) {
                                    reject = true;
                            }
                    } catch (UnsupportedOperationException e) {
                            // The task queue does not support removal so the best thing we can do is to just move on and
                            // hope we will be able to pick-up the task before its completely terminated.
                            // In worst case we will log on termination.
                    }
                    if (reject) {
                            reject();
                    }
            }
    }

    if (!addTaskWakesUp && wakesUpForTask(task)) {
            wakeup(inEventLoop);
    }
}

可以看到,在execute方法中,如果执行execute方法的线程不是在EventLoop线程中,那么会进行inEventLoop()判断,并且调用startThread()方法开启线程并进入doStartThread()方法中,可以看见:

executor.execute(new Runnable() {
        ...
});
executor为ThreadPerTaskExecutor,查看execute方法:
@Override
public void execute(Runnable command) {
        threadFactory.newThread(command).start();
}

因此可以看见,调用此方法后会创建一个新的线程,并且执行传进来的任务:具体任务为SingleThreadEventExecutor.this.run(),这个方法在NioEventLoop中实现:

protected void run() {
        for (;;) {
                try {
                        try {
                                //在此处会进行判断,如果TaskQueue中有任务,那么会调用selectNow()方法,否则会进入SelectStrategy.SELECT部分(调用select())
                                switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
                                case SelectStrategy.CONTINUE:
                                        continue;

                                case SelectStrategy.BUSY_WAIT:

                                case SelectStrategy.SELECT:
                                        select(wakenUp.getAndSet(false));
                                        if (wakenUp.get()) {
                                                selector.wakeup();
                                        }
                                        // fall through
                                default:
                                }
                        } catch (IOException e) {
                                rebuildSelector0();
                                handleLoopException(e);
                                continue;
                        }

                        cancelledKeys = 0;
                        needsToSelectAgain = false;

                        // ioRatio为IO任务和普通任务处理时间占比

                        final int ioRatio = this.ioRatio;
                        if (ioRatio == 100) {
                                try {
                                        //在此处处理IO事件
                                        processSelectedKeys();
                                } finally {
                                        // 在此处处理TaskQueue中的任务
                                        runAllTasks();
                                }
                        } else {
                                final long ioStartTime = System.nanoTime();
                                try {
                                        processSelectedKeys();
                                } finally {
                                        // Ensure we always run tasks.
                                        final long ioTime = System.nanoTime() - ioStartTime;
                                        runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
                                }
                        }
                } catch (Throwable t) {
                        handleLoopException(t);
                }
                // Always handle shutdown even if the loop processing threw an exception.
                try {
                        if (isShuttingDown()) {
                                closeAll();
                                if (confirmShutdown()) {
                                        return;
                                }
                        }
                } catch (Throwable t) {
                        handleLoopException(t);
                }
        }
}
	

上面的代码有三个部分需要细看:

select(wakenUp.getAndSet(false));
processSelectedKeys();
runAllTasks();
	

processSelectedKeys()方法中主要调用了private void processSelectedKey(SelectionKey k, AbstractNioChannel ch)方法其中k是select出来的有事件的socket,AbstractNioChannel则是从SelectedKeys中获取的attachment(上面说过,每一个java原生SocketChannel通过Selector的attachment关联着一个Netty的NioSocketChannel)

private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
        final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
        if (!k.isValid()) {
                final EventLoop eventLoop;
                try {
                        eventLoop = ch.eventLoop();
                } catch (Throwable ignored) {
                        // If the channel implementation throws an exception because there is no event loop, we ignore this
                        // because we are only trying to determine if ch is registered to this event loop and thus has authority
                        // to close ch.
                        return;
                }
                // Only close ch if ch is still registered to this EventLoop. ch could have deregistered from the event loop
                // and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
                // still healthy and should not be closed.
                // See https://github.com/netty/netty/issues/5125
                if (eventLoop != this || eventLoop == null) {
                        return;
                }
                // close the channel if the key is not valid anymore
                unsafe.close(unsafe.voidPromise());
                return;
        }

        try {
                int readyOps = k.readyOps();
                // We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
                // the NIO JDK channel implementation may throw a NotYetConnectedException.
                if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
                        // remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
                        // See https://github.com/netty/netty/issues/924
                        int ops = k.interestOps();
                        ops &= ~SelectionKey.OP_CONNECT;
                        k.interestOps(ops);

                        unsafe.finishConnect();
                }

                // Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
                if ((readyOps & SelectionKey.OP_WRITE) != 0) {
                        // Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
                        ch.unsafe().forceFlush();
                }

                // Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
                // to a spin loop
                if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
                        unsafe.read();
                }
        } catch (CancelledKeyException ignored) {
                unsafe.close(unsafe.voidPromise());
        }
}

可以看出,此方法调用了每个channel自己的unsafe方法处理了对应的Socket事件,例如SelectionKey.OP_WRITE,SelectionKey.OP_READ等

runAllTasks()比较简单

protected final boolean runAllTasksFrom(Queue<Runnable> taskQueue) {
        Runnable task = pollTaskFrom(taskQueue);
        if (task == null) {
                return false;
        }
        for (;;) {
                safeExecute(task);
                task = pollTaskFrom(taskQueue);
                if (task == null) {
                        return true;
                }
        }
}

protected static void safeExecute(Runnable task) {
        try {
                task.run();
        } catch (Throwable t) {
                logger.warn("A task raised an exception. Task: {}", task, t);
        }
}

由上可见,就是在此线程中依次执行taskQueue中的所有任务,到此,线程已启动,NioEventLoop已经开始运行,不断循环运行着run方法,直到关闭

现在回来说说,每个Channel在Register过程中是怎么做到切换线程执行register0的 在initAndRegister()方法中 ChannelFuture regFuture = config().group().register(channel); 这个方法拿到了NioEventLoopGroup,并调用了NioEventLoopGroup的register,实际上就是选择一个NioEventLoop并调用registerSingleThreadEventLoop类的方法

@Override
public ChannelFuture register(final ChannelPromise promise) {
        ObjectUtil.checkNotNull(promise, "promise");
        promise.channel().unsafe().register(this, promise);
        return promise;
}

这个ChannelPromise是new DefaultChannelPromise(channel, this),最后调用了新建的channel的unsafe的register方法

public final void register(EventLoop eventLoop, final ChannelPromise promise) {
        ......
        AbstractChannel.this.eventLoop = eventLoop;
        if (eventLoop.inEventLoop()) {
            register0(promise);
        } else {
            try {
                eventLoop.execute(new Runnable() {
                    @Override
                    public void run() {
                        register0(promise);
                    }
                });
            }
                            .....
                    }
                    .....
            }
	

这部分是连接SingleThreadEventExecutor的execute方法

public void execute(Runnable task) {
        ...
        boolean inEventLoop = inEventLoop();
        addTask(task);
        if (!inEventLoop) {
                startThread();
        }
        ...
        if (!addTaskWakesUp && wakesUpForTask(task)) {
                wakeup(inEventLoop);
        }
}

由这段代码分析:当第一次register调用时选择了一个新建的NioEventLoop来执行execute时,会先把任务加入NioEventLoop的taskQueue中,此时因为NioEventLoop中的thread为null,在inEventLoop()方法中返回false,因此会进入startThread(),又因为thread为null,因此会创建一个线程,最后执行wakeup(inEventLoop)唤醒当前NioEventLoop的线程来执行新进来的任务。

当第一次register调用时选择了一个已经建好的的NioEventLoop来执行execute时,会先把任务加入NioEventLoop的taskQueue中,此时因为NioEventLoop中的thread已经有值并且不为主线程,在inEventLoop()方法中返回false,因此会进入startThread(),又因为thread有值,因此会跳过此方法,最后执行wakeup(inEventLoop)唤醒当前NioEventLoop的线程来执行新进来的任务

五、netty bytebuf内存池

可以带着以下问题去看Netty内存池源码:

内存池管理算法是怎么做到申请效率,怎么减少内存碎片
高负载下内存池不断扩展,如何做到内存回收
对象池是如何实现的,这个不是关键路径,可以当成黑盒处理
内存池跟对象池作为全局数据,在多线程环境下如何减少锁竞争
池化后内存的申请跟释放必然是成对出现的,那么如何做内存泄漏检测,特别是跨线程之间的申请跟释放是如何处理的。

(待更新)