前言
这篇文章是Netty学习系列的第三篇文章，从上面一篇文章我们了解到NioEventLoop是Netty处理事件的核心机制。具体的我将本篇文章分为四个大的方面，NioEventLoop创建、NioEventLoop启动、NioEventLoop执行逻辑和NioEventLoop的总结。
一 NioEventLoop的创建
在创建NioEventLoop时首先会创建一个线程执行器，这个线程执行器的作用就是创建一个NioEventLoop对应底层的一个线程，然后会用一个for循环创建一个EventLoop数组，然后再创建每一个EventLoop的时候回调用一个 newChild() 方法，给NioEventLoop配置属性，在最后会创建一个选择器给每个新连接分配一个 NioEventLoop,具体的相关函数调用可以看下图：
还记得我们在第一篇文章的例子程序中服务端启动的时候，传入了两个EventLoopGroup如下：
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
由上面的代码可知，我们是在这里创建的NioEventLoop,我们跟进去看看他的构造函数，源码如下：
/**
* NioEventLoopGroup.java 36行起
*/
public class NioEventLoopGroup extends MultithreadEventLoopGroup {
/**
* Create a new instance using the default number of threads, the default {@link ThreadFactory} and
* the {@link SelectorProvider} which is returned by {@link SelectorProvider#provider()}.
*/
public NioEventLoopGroup() {
this(0);
}
/**
* Create a new instance using the specified number of threads, {@link ThreadFactory} and the
* {@link SelectorProvider} which is returned by {@link SelectorProvider#provider()}.
*/
public NioEventLoopGroup(int nThreads) {
this(nThreads, (Executor) null);
}
......
}
可以从这里知道，无参构造函数会自动给我们设置一个0,而去调用有参够着函数，这里我们继续跟下去看看他们的内部逻辑到底是怎样创建的，其实到最后他会调用到父类MultithreadEventLoopGroup的构造函数，具体源码如下：
/**
*MultithreadEventLoopGroup.java 50行起
*/
public abstract class MultithreadEventLoopGroup extends MultithreadEventExecutorGroup implements EventLoopGroup {
......
/**
* @see {@link MultithreadEventExecutorGroup#MultithreadEventExecutorGroup(int, Executor, Object...)}
*/
protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
}
......
}
在这里我们其实可以看到，当我们传0过来的时候会给我们默认赋一个值DEFAULT_EVENT_LOOP_THREADS这个字的大小就是cpu核数的两倍理论上的最佳线程数，另外可知这里也不是真正创建的地方，我们继续跟进代码，源码如下：
/**
* MultithreadEventExecutorGroup.java 30行起
*/
public abstract class MultithreadEventExecutorGroup extends AbstractEventExecutorGroup {
......
/**
* Create a new instance.
*
* @param nThreads the number of threads that will be used by this instance.
* @param executor the Executor to use, or {@code null} if the default should be used.
* @param args arguments which will passed to each {@link #newChild(Executor, Object...)} call
*/
protected MultithreadEventExecutorGroup(int nThreads, Executor executor, Object... args) {
this(nThreads, executor, DefaultEventExecutorChooserFactory.INSTANCE, args);
}
/**
* Create a new instance.
*
* @param nThreads the number of threads that will be used by this instance.
* @param executor the Executor to use, or {@code null} if the default should be used.
* @param chooserFactory the {@link EventExecutorChooserFactory} to use.
* @param args arguments which will passed to each {@link #newChild(Executor, Object...)} call
*/
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
EventExecutorChooserFactory chooserFactory, Object... args) {
if (nThreads <= 0) {
throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
}
if (executor == null) {
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
}
children = new EventExecutor[nThreads];
for (int i = 0; i < nThreads; i ++) {
boolean success = false;
try {
children[i] = newChild(executor, args);
success = true;
} catch (Exception e) {
// TODO: Think about if this is a good exception type
throw new IllegalStateException("failed to create a child event loop", e);
} finally {
if (!success) {
for (int j = 0; j < i; j ++) {
children[j].shutdownGracefully();
}
for (int j = 0; j < i; j ++) {
EventExecutor e = children[j];
try {
while (!e.isTerminated()) {
e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
}
} catch (InterruptedException interrupted) {
// Let the caller handle the interruption.
Thread.currentThread().interrupt();
break;
}
}
}
}
}
chooser = chooserFactory.newChooser(children);
final FutureListener<Object> terminationListener = new FutureListener<Object>() {
@Override
public void operationComplete(Future<Object> future) throws Exception {
if (terminatedChildren.incrementAndGet() == children.length) {
terminationFuture.setSuccess(null);
}
}
};
for (EventExecutor e: children) {
e.terminationFuture().addListener(terminationListener);
}
Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);
Collections.addAll(childrenSet, children);
readonlyChildren = Collections.unmodifiableSet(childrenSet);
}
......
}
简单的分析可知下面的那个构造方法才是真正创建的地方，在这里我们把这部分代码拆分分析。
1.1 创建线程选择器
这部分主要是创建一个ThreadPerTaskExecutor，对应的代码片段是，如下：
if (executor == null) {
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
}
我们继续跟进这个代码，如下所示：
public final class ThreadPerTaskExecutor implements Executor {
private final ThreadFactory threadFactory;
public ThreadPerTaskExecutor(ThreadFactory threadFactory) {
if (threadFactory == null) {
throw new NullPointerException("threadFactory");
}
this.threadFactory = threadFactory;
}
@Override
public void execute(Runnable command) {
threadFactory.newThread(command).start();
}
}
在上面的代码中可以看到这个线程执行器的作用就是每次执行任务就调用execute()方法，这个方法的本质就是用传进来的threadFactory创建一个线程并把任务放在里面，然后执行。下面我们继续分析传进来这个线程工厂(由 newDefaultThreadFactory() 创建)都干了啥，具体的源码如下：
/**
* DefaultThreadFactory.java
*/
protected ThreadFactory newDefaultThreadFactory() {
return new DefaultThreadFactory(getClass());
}
组织这里我们将NioEventLoop这个类当参数传进去，我们继续跟进源码，可以发现会调用到DefaultThreadFactory这个方法，如下：
public DefaultThreadFactory(Class<?> poolType, boolean daemon, int priority) {
this(toPoolName(poolType), daemon, priority);
}
public static String toPoolName(Class<?> poolType) {
if (poolType == null) {
throw new NullPointerException("poolType");
}
String poolName = StringUtil.simpleClassName(poolType);
switch (poolName.length()) {
case 0:
return "unknown";
case 1:
return poolName.toLowerCase(Locale.US);
default:
if (Character.isUpperCase(poolName.charAt(0)) && Character.isLowerCase(poolName.charAt(1))) {
return Character.toLowerCase(poolName.charAt(0)) + poolName.substring(1);
} else {
return poolName;
}
}
}
有上面的代码可以知道其实还没有到最终调用，但是我们这里要分析一下这个传参函数toPoolName，其实分析这个函数就知道这是在给线程词创建名字具体做的逻辑就是将首字母小写，好了我们继续看看下面的调用逻辑，具体代码如下：
/**
* DefaultThreadFactory.java
*/
public DefaultThreadFactory(String poolName, boolean daemon, int priority, ThreadGroup threadGroup) {
if (poolName == null) {
throw new NullPointerException("poolName");
}
if (priority < Thread.MIN_PRIORITY || priority > Thread.MAX_PRIORITY) {
throw new IllegalArgumentException(
"priority: " + priority + " (expected: Thread.MIN_PRIORITY <= priority <= Thread.MAX_PRIORITY)");
}
prefix = poolName + '-' + poolId.incrementAndGet() + '-';
this.daemon = daemon;
this.priority = priority;
this.threadGroup = threadGroup;
}
这里的prefix就是我们到时候创建爱线程的名字前缀，可以知道这个前缀就是我们上一步得到的线程池的名字加上“-”再加上poolId.incrementAndGet() 这个函数的返回值，这个函数是java的util包中的一个函数，就是得到线程的一个自增编号，总结现在的 prefix就等于nioEventLoop-x-，下面我们回到线程池的newThread方法，这个方法也是线程选择器中调用的方法，具体的源码如下：
@Override
/**
* DefaultThreadFactory.java
*/
public Thread newThread(Runnable r) {
Thread t = newThread(new DefaultRunnableDecorator(r), prefix + nextId.incrementAndGet());
try {
if (t.isDaemon()) {
if (!daemon) {
t.setDaemon(false);
}
} else {
if (daemon) {
t.setDaemon(true);
}
}
if (t.getPriority() != priority) {
t.setPriority(priority);
}
} catch (Exception ignored) {
// Doesn't matter even if failed to set.
}
return t;
}
protected Thread newThread(Runnable r, String name) {
return new FastThreadLocalThread(threadGroup, r, name);
}
这里可以看出，这个方法最终是new了一个FastThreadLocalThread，这个不是jdk原生的Thread,也可以清晰的看到给这个线程创建的名字就是我们最先分析的到的prefix加上线程的编号，其实这个FastThreadLocalThread是继承原生Thread的，这里实际上他是对ThreadLocal进行了一个优化，这里我就不详细分析了。总结一下这个小节就是，每次执行任务就会创建一个线程实体，并且根据定义的一套命名规则命名。
1.2 创建EventLoop线程
这一小节的代码主要体现在newChild上面，我们也主要分析这个方法，他在源码中的体现是如下：
try {
children[i] = newChild(executor, args);
success = true;
} catch (Exception e) {
// TODO: Think about if this is a good exception type
throw new IllegalStateException("failed to create a child event loop", e);
}
我们跟进去看看newChild的具体实现,newChild是一个抽象函数，我们这里分析NioEventLoopGroup的实现，具体源码如下：
/**
* NioEventLoopGroup.java
*/
@Override
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
return new NioEventLoop(this, executor, (SelectorProvider) args[0],
((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]);
}
看到这里我们继续跟源码：
/**
* EventLoop.java 138行起
*/
NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
if (selectorProvider == null) {
throw new NullPointerException("selectorProvider");
}
if (strategy == null) {
throw new NullPointerException("selectStrategy");
}
provider = selectorProvider;
selector = openSelector();
selectStrategy = strategy;
}
看到这里我们先放放，先去看看父类构造函数在干什么，具体的源码如下：
/**
* SingleThreadEventExecutor.java 172行起
*/
protected SingleThreadEventExecutor(EventExecutorGroup parent, Executor executor,
boolean addTaskWakesUp, int maxPendingTasks,
RejectedExecutionHandler rejectedHandler) {
super(parent);
this.addTaskWakesUp = addTaskWakesUp;
this.maxPendingTasks = Math.max(16, maxPendingTasks);
this.executor = ObjectUtil.checkNotNull(executor, "executor");
taskQueue = newTaskQueue(this.maxPendingTasks);
rejectedExecutionHandler = ObjectUtil.checkNotNull(rejectedHandler, "rejectedHandler");
}
在这里就是父类构造函数，主要就是保存线程执行器ThreadPerTaskExecutor还有创建了一个MpscQueue，这个MpscQueue就是保存要执行的任务。好了我们继续回到子类的够着函数，我们可以看到有一句selector = openSelector()，这其实创建一个selector并赋值给当前类。
1.3 创建线程选择器
这一小节主要体现在 chooserFactory.newChooser(children) 这个方法上，这个作用主要是给新连接绑定EventLoop,绑定时主要是调用EventExecutorChooser的next方法，这个next方法要实现的需求就是一个循环数组，从0取到n,然后又继续取0。好了我们去看看这个newChooser方法吧，具体的源码如下：
/**
* DefaultEventExecutorChooserFactory.java 34行起
*/
@Override
public EventExecutorChooser newChooser(EventExecutor[] executors) {
if (isPowerOfTwo(executors.length)) {
return new PowerOfTowEventExecutorChooser(executors);
} else {
return new GenericEventExecutorChooser(executors);
}
}
我们分析这里，isPowerOfTwo这个方法是判断这个数组length是否是2的倍数，为什么要这样子判断，其实是Netty做的一个小优化，具体的我们看后面面的分析，这里我们先看看不是2的幂的时候创建的GenericEventExecutorChooser，具体的源码如下：
/**
* DefaultEventExecutorChooserFactory.java 60行起
*/
private static final class GenericEventExecutorChooser implements EventExecutorChooser {
private final AtomicInteger idx = new AtomicInteger();
private final EventExecutor[] executors;
GenericEventExecutorChooser(EventExecutor[] executors) {
this.executors = executors;
}
@Override
public EventExecutor next() {
return executors[Math.abs(idx.getAndIncrement() % executors.length)];
}
}
这里看到在next方法中，实现循环数组的方式就是普通的取余操作，这里就不过多分许，下面我们看看PowerOfTowEventExecutorChooser这个内部的实现，具体的源码如下：
/**
* DefaultEventExecutorChooserFactory.java 46行起
*/
private static final class PowerOfTowEventExecutorChooser implements EventExecutorChooser {
private final AtomicInteger idx = new AtomicInteger();
private final EventExecutor[] executors;
PowerOfTowEventExecutorChooser(EventExecutor[] executors) {
this.executors = executors;
}
@Override
public EventExecutor next() {
return executors[idx.getAndIncrement() & executors.length - 1];
}
}
这里还是看到在next方法中，实现循环数组的方式就变了刚才的取余操作变成了&操作，这样子虽然是一个小的优化，但是实际的效率还是有提高的。至此我们吧NioEventLoop的创建就分析完了，下一节我们将介绍NioEventLoop的启动。
二 NioEventLoop的启动
NioEventLoop的启动其实我们在系列文章的第二篇文章中分析过，具体的源码如下：
/**
* AbstractBootstrap.java 335行起
*/
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}
在上一篇文章中我们是直接分析的channel.bind这个方法，而忽略了channel.eventLoop().execute这个方法的分析，这里我们需要分析的就是这个方法：
/**
* SingleThreadEventExecutor.java 751行起
*/
@Override
public void execute(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
boolean inEventLoop = inEventLoop();
if (inEventLoop) {
addTask(task);
} else {
startThread();
addTask(task);
if (isShutdown() && removeTask(task)) {
reject();
}
}
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}
在这里**inEventLoop()**这个方法是判断是否是在EventLoop的执行线程，可知这里我们是在主线程中的，所以这里应该是false,那我们继续分析下面的源码,先看看 **startThread()**这个方法干了啥吧，具体的源码如下：
/**
* SingleThreadEventExecutor.java 852行起
*/
private void startThread() {
if (STATE_UPDATER.get(this) == ST_NOT_STARTED) {
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
doStartThread();
}
}
}
在这里是先判断这个线程是否启动了，如果没启动就通过一个compareAndSet去实际的启动线程，启动成功后就会去调用**doStartThread()**方法，具体的源码如下：
/**
* SingleThreadEventExecutor.java 860行起
*/
private void doStartThread() {
assert thread == null;
executor.execute(new Runnable() {
@Override
public void run() {
thread = Thread.currentThread();
if (interrupted) {
thread.interrupt();
}
boolean success = false;
updateLastExecutionTime();
try {
SingleThreadEventExecutor.this.run();
success = true;
} catch (Throwable t) {
logger.warn("Unexpected exception from an event executor: ", t);
} finally {
for (;;) {
int oldState = STATE_UPDATER.get(SingleThreadEventExecutor.this);
if (oldState >= ST_SHUTTING_DOWN || STATE_UPDATER.compareAndSet(
SingleThreadEventExecutor.this, oldState, ST_SHUTTING_DOWN)) {
break;
}
}
// Check if confirmShutdown() was called at the end of the loop.
if (success && gracefulShutdownStartTime == 0) {
logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " +
SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must be called " +
"before run() implementation terminates.");
}
try {
// Run all remaining tasks and shutdown hooks.
for (;;) {
if (confirmShutdown()) {
break;
}
}
} finally {
try {
cleanup();
} finally {
STATE_UPDATER.set(SingleThreadEventExecutor.this, ST_TERMINATED);
threadLock.release();
if (!taskQueue.isEmpty()) {
logger.warn(
"An event executor terminated with " +
"non-empty task queue (" + taskQueue.size() + ')');
}
terminationFuture.setSuccess(null);
}
}
}
}
});
}
在这里其实doStartThread就是调用我们上面一节分析的线程执行器的execute方法，而这个executor实际就是我们创建线程执行器是创建的。而这个任务其实就是先保存当前线程到EventLoop中，具体的代码体现就是thread = Thread.currentThread()，接下来调用EventLoop的run()，做司机的执行操作，这个司机的执行操作下一节会详细介绍，NioEventLoop的启动我们就暂时分析在这里，下面一节我们会分析真正的执行过程。
三 NioEventLoop的执行
从上一小节可知NioEventLoop的执行主要是在其run方法里面，我们进入源码看看里面的具体实现是什么，具体源码如下：
/**
* NioEventLoop.java 393行起
*/
@Override
protected void run() {
for (;;) {
try {
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.SELECT:
select(wakenUp.getAndSet(false));
// 'wakenUp.compareAndSet(false, true)' is always evaluated
// before calling 'selector.wakeup()' to reduce the wake-up
// overhead. (Selector.wakeup() is an expensive operation.)
//
// However, there is a race condition in this approach.
// The race condition is triggered when 'wakenUp' is set to
// true too early.
//
// 'wakenUp' is set to true too early if:
// 1) Selector is waken up between 'wakenUp.set(false)' and
// 'selector.select(...)'. (BAD)
// 2) Selector is waken up between 'selector.select(...)' and
// 'if (wakenUp.get()) { ... }'. (OK)
//
// In the first case, 'wakenUp' is set to true and the
// following 'selector.select(...)' will wake up immediately.
// Until 'wakenUp' is set to false again in the next round,
// 'wakenUp.compareAndSet(false, true)' will fail, and therefore
// any attempt to wake up the Selector will fail, too, causing
// the following 'selector.select(...)' call to block
// unnecessarily.
//
// To fix this problem, we wake up the selector again if wakenUp
// is true immediately after selector.select(...).
// It is inefficient in that it wakes up the selector for both
// the first case (BAD - wake-up required) and the second case
// (OK - no wake-up required).
if (wakenUp.get()) {
selector.wakeup();
}
default:
// fallthrough
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
上面的代码还是挺复杂的，但是我把这里可以这里分为三部分，首先是调用select方法检查是否有io事件，然后调用 processSelectedKeys() 处理在上一过程中轮询出来的io事件，最后调用 **runAllTasks()**来处理异步任务队列中的任务，具体的可以用下面一幅图来展现，如图：
3.1 检测IO事件
由上面可知，检测IO事件是在select方法中实现的，我们这里县具体的看看这个方法的源码，源码如下：
/**
* NioEventLoop.java 733行起
*/
private void select(boolean oldWakenUp) throws IOException {
Selector selector = this.selector;
try {
int selectCnt = 0;
long currentTimeNanos = System.nanoTime();
long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
for (;;) {
long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
if (timeoutMillis <= 0) {
if (selectCnt == 0) {
selector.selectNow();
selectCnt = 1;
}
break;
}
// If a task was submitted when wakenUp value was true, the task didn't get a chance to call
// Selector#wakeup. So we need to check task queue again before executing select operation.
// If we don't, the task might be pended until select operation was timed out.
// It might be pended until idle timeout if IdleStateHandler existed in pipeline.
if (hasTasks() && wakenUp.compareAndSet(false, true)) {
selector.selectNow();
selectCnt = 1;
break;
}
int selectedKeys = selector.select(timeoutMillis);
selectCnt ++;
if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks() || hasScheduledTasks()) {
// - Selected something,
// - waken up by user, or
// - the task queue has a pending task.
// - a scheduled task is ready for processing
break;
}
if (Thread.interrupted()) {
// Thread was interrupted so reset selected keys and break so we not run into a busy loop.
// As this is most likely a bug in the handler of the user or it's client library we will
// also log it.
//
// See https://github.com/netty/netty/issues/2426
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely because " +
"Thread.currentThread().interrupt() was called. Use " +
"NioEventLoop.shutdownGracefully() to shutdown the NioEventLoop.");
}
selectCnt = 1;
break;
}
long time = System.nanoTime();
if (time - TimeUnit.MILLISECONDS.toNanos(timeoutMillis) >= currentTimeNanos) {
// timeoutMillis elapsed without anything selected.
selectCnt = 1;
} else if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
// The selector returned prematurely many times in a row.
// Rebuild the selector to work around the problem.
logger.warn(
"Selector.select() returned prematurely {} times in a row; rebuilding Selector {}.",
selectCnt, selector);
rebuildSelector();
selector = this.selector;
// Select again to populate selectedKeys.
selector.selectNow();
selectCnt = 1;
break;
}
currentTimeNanos = time;
}
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
}
} catch (CancelledKeyException e) {
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
selector, e);
}
// Harmless exception - log anyway
}
}
首先是deadline以及任务穿插逻辑处理，这里当我们调用select方法是这样子调用的，select(wakenUp.getAndSet(false))，这里每次进行select的时候都把唤醒操作设置成false传入进来。这部分对应的逻辑在源码中是如下代码体现：
int selectCnt = 0;
long currentTimeNanos = System.nanoTime();
long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
for (;;) {
long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
if (timeoutMillis <= 0) {
if (selectCnt == 0) {
selector.selectNow();
selectCnt = 1;
}
break;
}
// If a task was submitted when wakenUp value was true, the task didn't get a chance to call
// Selector#wakeup. So we need to check task queue again before executing select operation.
// If we don't, the task might be pended until select operation was timed out.
// It might be pended until idle timeout if IdleStateHandler existed in pipeline.
if (hasTasks() && wakenUp.compareAndSet(false, true)) {
selector.selectNow();
selectCnt = 1;
break;
}
首先进来的时候是先计算当前的时间，然后当前时间加上一个延迟时间，其中**delayNanos()**这个方法就是计算当前时间，在定时队列中的第一个截止时间，这两个时间相加得到一个截止时间。下面的逻辑就是进入一个for循环，进入for循环，首先是计算当前是否超时，如果超时切一次也没有select那么就会执行一个非阻塞的select方法，具体的代码体现是 selector.selectNow()，然后就break那么本次的select就结束。那么如果没有到截止时间，先检查当前异步队列里面是否有任务具体的是用 hasTasks()这个方法来判断的，然后因为刚进来是传入的wakenUp是false这里调用compareAndSet方法就会更新为true这里，就会执行非阻塞的select方法。这上面就是Select操作的第一个过程，第二个过程就是一个 阻塞式的select,具体的源码如下：
int selectedKeys = selector.select(timeoutMillis);
selectCnt ++;
if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks() || hasScheduledTasks()) {
// - Selected something,
// - waken up by user, or
// - the task queue has a pending task.
// - a scheduled task is ready for processing
break;
}
这里就是如果当前截止时间未到，并且任务队列为空就会进行一个阻塞式的select操作，这里的timeoutMillis就是能够阻塞的最大时间。每次select后就把这个selectCnt加一，表示Select轮询添加了一次。接下来判断如果轮询到了一个时间，或者当前select操作是否需要唤醒，或者在执行select操作的时候已经被外部线程唤醒，或者异步队列里面有任务了，又或者当前定时任务队列里面是否有任务，只要满足这几个条件之一本次select操作就中止。最后一个过程就是为了避免jdk空轮询bug，具体的源码体现如下：
long time = System.nanoTime();
if (time - TimeUnit.MILLISECONDS.toNanos(timeoutMillis) >= currentTimeNanos) {
// timeoutMillis elapsed without anything selected.
selectCnt = 1;
} else if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
// The selector returned prematurely many times in a row.
// Rebuild the selector to work around the problem.
logger.warn(
"Selector.select() returned prematurely {} times in a row; rebuilding Selector {}.",
selectCnt, selector);
rebuildSelector();
selector = this.selector;
// Select again to populate selectedKeys.
selector.selectNow();
selectCnt = 1;
break;
}
currentTimeNanos = time;
}
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
}
} catch (CancelledKeyException e) {
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
selector, e);
}
// Harmless exception - log anyway
}
执行完一次阻塞式的select操作后，会记录下当前时间。然后用当前时间减去开始时间大于超时时间就将selectCnt置为1，就是表明已经执行了一次阻塞式的select操作，或者判断空轮询次数大于SELECTOR_AUTO_REBUILD_THRESHOLD这个值(默认为512)，就会调用一个rebuildSelector() 方法，这个方法的作用就是把老的Selector上的所有Selector.Key,注册到一个新的Selector上去，这样新的Selector中的阻塞式操作就有可能不会发生空轮询bug，这里的具体实现逻辑就不过多的介绍了。上面替换完后就会调用**selectNow()**重新进行一次select操作。在这里检测IO事件就分析完了，下面小节我们就介绍处理IO事件。
3.2 处理IO事件
在这个章节我主要介绍两个方面，一个方面Netty在底层selected keySet优化，select操作每次都会把已经就绪状态的事件添加到一个底层的hashSet集合中，而netty会通过反射的方式将hashSet替换成数组的实现，这样在任何情况下select的操作时间复杂度都是O(1)，优于hashset。另一方面就是去调用**processSelectedKeysOptimized()**方法，去真的处理这些IO事件。下面我们分别来分析，首先在前面的章节我们知道在创建NioEventLoop的过程中会调用一个 **openSelector()**方法去创建底层的一个一一对应的 selector，具体的代码如下：
NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
if (selectorProvider == null) {
throw new NullPointerException("selectorProvider");
}
if (strategy == null) {
throw new NullPointerException("selectStrategy");
}
provider = selectorProvider;
selector = openSelector();
selectStrategy = strategy;
}
private Selector openSelector() {
final Selector selector;
try {
selector = provider.openSelector();
} catch (IOException e) {
throw new ChannelException("failed to open a new selector", e);
}
if (DISABLE_KEYSET_OPTIMIZATION) {
return selector;
}
final SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet();
Object maybeSelectorImplClass = AccessController.doPrivileged(new PrivilegedAction<Object>() {
@Override
public Object run() {
try {
return Class.forName(
"sun.nio.ch.SelectorImpl",
false,
PlatformDependent.getSystemClassLoader());
} catch (ClassNotFoundException e) {
return e;
} catch (SecurityException e) {
return e;
}
}
});
if (!(maybeSelectorImplClass instanceof Class) ||
// ensure the current selector implementation is what we can instrument.
!((Class<?>) maybeSelectorImplClass).isAssignableFrom(selector.getClass())) {
if (maybeSelectorImplClass instanceof Exception) {
Exception e = (Exception) maybeSelectorImplClass;
logger.trace("failed to instrument a special java.util.Set into: {}", selector, e);
}
return selector;
}
final Class<?> selectorImplClass = (Class<?>) maybeSelectorImplClass;
Object maybeException = AccessController.doPrivileged(new PrivilegedAction<Object>() {
@Override
public Object run() {
try {
Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");
selectedKeysField.setAccessible(true);
publicSelectedKeysField.setAccessible(true);
selectedKeysField.set(selector, selectedKeySet);
publicSelectedKeysField.set(selector, selectedKeySet);
return null;
} catch (NoSuchFieldException e) {
return e;
} catch (IllegalAccessException e) {
return e;
} catch (RuntimeException e) {
// JDK 9 can throw an inaccessible object exception here; since Netty compiles
// against JDK 7 and this exception was only added in JDK 9, we have to weakly
// check the type
if ("java.lang.reflect.InaccessibleObjectException".equals(e.getClass().getName())) {
return e;
} else {
throw e;
}
}
}
});
if (maybeException instanceof Exception) {
selectedKeys = null;
Exception e = (Exception) maybeException;
logger.trace("failed to instrument a special java.util.Set into: {}", selector, e);
} else {
selectedKeys = selectedKeySet;
logger.trace("instrumented a special java.util.Set into: {}", selector);
}
return selector;
}
我们这里跟一下这个openSelector方法，首先会调用jdk的openSelector创建一个原生的selector,然后判断是否需要优化，如果不需要优化就直接返回原生的selector,反之则进行优化，下面我们看看优化步骤。首先我们会创建一个SelectedSelectionKeySet的数据结构用来替换原生selector中的selectedKeySet,在这里我们县看看这个数据结构内部的代码吧，源码如下：
/**
* SelectedSelectionKeySet.java
*/
final class SelectedSelectionKeySet extends AbstractSet<SelectionKey> {
private SelectionKey[] keysA;
private int keysASize;
private SelectionKey[] keysB;
private int keysBSize;
private boolean isA = true;
SelectedSelectionKeySet() {
keysA = new SelectionKey[1024];
keysB = keysA.clone();
}
@Override
public boolean add(SelectionKey o) {
if (o == null) {
return false;
}
if (isA) {
int size = keysASize;
keysA[size ++] = o;
keysASize = size;
if (size == keysA.length) {
doubleCapacityA();
}
} else {
int size = keysBSize;
keysB[size ++] = o;
keysBSize = size;
if (size == keysB.length) {
doubleCapacityB();
}
}
return true;
}
private void doubleCapacityA() {
SelectionKey[] newKeysA = new SelectionKey[keysA.length << 1];
System.arraycopy(keysA, 0, newKeysA, 0, keysASize);
keysA = newKeysA;
}
private void doubleCapacityB() {
SelectionKey[] newKeysB = new SelectionKey[keysB.length << 1];
System.arraycopy(keysB, 0, newKeysB, 0, keysBSize);
keysB = newKeysB;
}
SelectionKey[] flip() {
if (isA) {
isA = false;
keysA[keysASize] = null;
keysBSize = 0;
return keysA;
} else {
isA = true;
keysB[keysBSize] = null;
keysASize = 0;
return keysB;
}
}
@Override
public int size() {
if (isA) {
return keysASize;
} else {
return keysBSize;
}
}
@Override
public boolean remove(Object o) {
return false;
}
@Override
public boolean contains(Object o) {
return false;
}
@Override
public Iterator<SelectionKey> iterator() {
throw new UnsupportedOperationException();
}
}
可以看出这个数据结构的内部实现是数组，而且这个数据结构是继承至AbstractSet,这里我们看他实现的父类函数只实现了add操作，而且这个add()内部的实现是基于数组的，这样时间复杂度就可以底至O(1),然后remove,contains,iterator这三个方法是默认不支持的，其实在使用时根本不需要这三个方法。好了这个数据结构我们先分析到这里，下面我们继续分析openSelector方法中的逻辑,接下来，我们会通过反射去得到一个SelectorImpl对象，后面就是各种判断是否拿到了这个类，如果没拿到我们还是会返回原生的Selector，如果拿到了我们还是通过反射的方式去调用如下两个方法：
selectedKeysField.set(selector, selectedKeySet);
publicSelectedKeysField.set(selector, selectedKeySet);
用我们优化好的selectedKeySet去替换原生的selectedKeySet。在这里我们就将第一部分的优化讲完了，下面我们分析真正开始处理IO事件。下面我们回到EventLoop的run方法，这里里面有这样一段逻辑，具体的代码如下：
/**
* NioEventLoop.java 448行起
*/
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
这里我们调用了 processSelectedKeys() 方法，我们跟进源码看看：
/**
* NioEventLoop.java 485行起
*/
private void processSelectedKeys() {
if (selectedKeys != null) {
processSelectedKeysOptimized(selectedKeys.flip());
} else {
processSelectedKeysPlain(selector.selectedKeys());
}
}
在这里首先判断我们经过优化后的selectedKeys是否为空，这里我们只分析不为空的情况吧，首先selectedKeys.flip() 会返回内部的数组，然后我们跟进processSelectedKeysOptimized 源码，如下：
/**
* NioEventLoop.java 561行起
*/
private void processSelectedKeysOptimized(SelectionKey[] selectedKeys) {
for (int i = 0;; i ++) {
final SelectionKey k = selectedKeys[i];
if (k == null) {
break;
}
// null out entry in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys[i] = null;
final Object a = k.attachment();
if (a instanceof AbstractNioChannel) {
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
if (needsToSelectAgain) {
// null out entries in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
for (;;) {
i++;
if (selectedKeys[i] == null) {
break;
}
selectedKeys[i] = null;
}
selectAgain();
// Need to flip the optimized selectedKeys to get the right reference to the array
// and reset the index to -1 which will then set to 0 on the for loop
// to start over again.
//
// See https://github.com/netty/netty/issues/1523
selectedKeys = this.selectedKeys.flip();
i = -1;
}
}
}
在这里面我们的主要逻辑是在processSelectedKey这个方法中，我们继续跟进源码，如下：
/**
* NioEventLoop.java 604行起
*/
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
// If the channel implementation throws an exception because there is no event loop, we ignore this
// because we are only trying to determine if ch is registered to this event loop and thus has authority
// to close ch.
return;
}
// Only close ch if ch is still registerd to this EventLoop. ch could have deregistered from the event loop
// and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
// still healthy and should not be closed.
// See https://github.com/netty/netty/issues/5125
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
if (!ch.isOpen()) {
// Connection already closed - no need to handle write.
return;
}
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}
首先我们会判断这个key的合法性，如果不合法我们会调用 unsafe.close() 方法关闭连接。，如果是合法的后面就是拿到IO事件，处理各种IO事件。
3.3 线程任务的执行
这是最后一部分，主要就是Task的执行，具体在NioEventLoop的的代码体现就是执行run方法时调用runAllTasks() 方法。在分析这个方法前我们得知道在Netty中有两种任务队列，一种是普通的任务队列，一种是定时的任务队列。下面我们开始分析这个方法，具体源码如下：
/**
* SingleThreadEventExecutor.java 406行起
*/
protected boolean runAllTasks(long timeoutNanos) {
fetchFromScheduledTaskQueue();
Runnable task = pollTask();
if (task == null) {
afterRunningAllTasks();
return false;
}
final long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
long runTasks = 0;
long lastExecutionTime;
for (;;) {
safeExecute(task);
runTasks ++;
// Check timeout every 64 tasks because nanoTime() is relatively expensive.
// XXX: Hard-coded value - will make it configurable if it is really a problem.
if ((runTasks & 0x3F) == 0) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
if (lastExecutionTime >= deadline) {
break;
}
}
task = pollTask();
if (task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
afterRunningAllTasks();
this.lastExecutionTime = lastExecutionTime;
return true;
}
然后我们可以发现先调用了fetchFromScheduledTaskQueue，这个方法，这个方法的作用就是讲任务给聚合-将定时任务队列插入当前任务队列，具体的源码如下：
/**
* SingleThreadEventExecutor.java 287行起
*/
private boolean fetchFromScheduledTaskQueue() {
long nanoTime = AbstractScheduledEventExecutor.nanoTime();
Runnable scheduledTask = pollScheduledTask(nanoTime);
while (scheduledTask != null) {
if (!taskQueue.offer(scheduledTask)) {
// No space left in the task queue add it back to the scheduledTaskQueue so we pick it up again.
scheduledTaskQueue().add((ScheduledFutureTask<?>) scheduledTask);
return false;
}
scheduledTask = pollScheduledTask(nanoTime);
}
return true;
}
这段代码的逻辑就是冲当前的定时任务队列中去拉取第一个任务，而这里定时任务中的排序是遵循一定的逻辑的，首先截止时间小的排在前面，如果截止时间相同的则序号小的排在前面。这里如果定时任务不为空则直接添加到普通任务队列中，在添加过程中如果失败，则重新将定时任务添加到定时任务队列中。后面则调用一个while循环继续冲定时任务队列中拉取定时任务，在这里任务的聚合就完成了。下面我们继续回到 runAllTasks() 方法中，在下面我们会通过 pollTask() 从聚合的任务队列中拿到一个任务，然后经过一系列的判断，最后在 safeExecute(task) 方法中执行任务的run方法。下面还有个重点，我们看一段代码如下：
// Check timeout every 64 tasks because nanoTime() is relatively expensive.
// XXX: Hard-coded value - will make it configurable if it is really a problem.
if ((runTasks & 0x3F) == 0) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
if (lastExecutionTime >= deadline) {
break;
}
}
task = pollTask();
if (task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
当累计到64是任务数时会计算一个当前时间，然后比较是否超过截止时间，如果超过了就break，这里为什么积累到64次才计算当前时间因为，其实计算这个当前时间也是一个耗时操作。如果没超过就继续执行任务，在这里我们就把这部分内容分析完了。
四 总结
下面我对本文做一个总结
- 用户代码再创建boosGroup和workGroup的时候，NioEventLoop被创建，默认不穿参数的时候，创建2倍CPU核心数的NioEventLoop
- 每一个NioEventLoop都会又一个chooser进行线程逻辑的分配，该chooser也会针对NioEventLoop的个数进行一些优化。
- 创建NioEventLoop的时候会创建一个selector和一个定时任务队列scheduledTaskQueue,并且创建selector的时候，创建了一个数组实现的SelectedSelectionKeySet替换掉原有HashSet实现的两个属性 selectedKeys和publicSelectedKeys
- NioEventLoop再首次调用execute()方法的时候启动FastThreadLocalThread线程，并保存到成员变量，这样就可以判断执行NioEventLoop执行逻辑的线程是否是当前线程。
- NioEventLoop的执行逻辑再run()方法，包括三个过程，分别是： （1）检测IO事件 （2）处理这些IO事件 （3）执行任务队列
参考资料
《Netty实战》 《Netty权威指南》《Netty深入剖析》
注：这里使用的是Netty4.1.6的源码进行分析
原文地址(我得个人博客):Netty学习系列(三)-EventLoop
