概述:本文将详细讲述netty中的EventLoop,会分为下面十部分:
- 一. NioEventLoop概述
- 二. NioEventLoop创建概述
- 三. ThreadPerTaskThread
- 四. 创建NioEventLoop线程
- 五. 创建线程选择器
- 六. NioEventLoop的启动
- 七. NioEventLoop执行概述
- 八. 检测IO事件
- 九.处理IO事件
- 十. reactor线程任务的执行
分析之前,还是看netty的启动模板代码 public class MyServer {
public static void main(String[] args) {
// 1.构建 bossGroup 和 workGroup
NioEventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workGroup = new NioEventLoopGroup(1);
ServerBootstrap serverBootstrap = new ServerBootstrap();
// 2.设置group
serverBootstrap.group(bossGroup, workGroup);
// 3.设置服务端option和attr
serverBootstrap.option(ChannelOption.SO_BACKLOG, 1);
serverBootstrap.attr(AttributeKey.valueOf("name"),"彭青松");
// 4.设置服务端的channel
serverBootstrap.channel(NioServerSocketChannel.class);
// 5.设置服务端的handler
serverBootstrap.handler(new MyServerHandler());
// 6.设置客户端的handler
serverBootstrap.childHandler(new ChannelInitializer() {
@Override
protected void initChannel(Channel ch) throws Exception {
// 7.客户端channel初始化的时候,调用这里
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast(new MyTcpClinetHandler());
}
});
try {
// 8.服务绑定端口,启动
ChannelFuture channelFuture = serverBootstrap.bind(8888).sync();
channelFuture.channel().closeFuture().sync();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
bossGroup.shutdownGracefully();
workGroup.shutdownGracefully();
}
}
}
一. NioEventLoop概述
从上面代码中可以看到,netty启动的第一步,就是设置两个bossGroup和workGroup, 每一个NioEventLoopGroup包含一个/多个NioEventLoop,每一个NioEventLoop是一个线程,NioEventLoop中包含一个Selector,channel创建后会把自己注册到selector上,并设置关注的事件,然后启动这个NioEventLoop,读取IO事件,处理IO事件,处理任务队列中的任务,如下图:
二. NioEventLoop创建
NioEventLoopGroup bossGroup = new NioEventLoopGroup(1);
紧着进入
public NioEventLoopGroup(int nThreads) {
this(nThreads, (Executor) null);
}
接下来会进入几个重载方法,最后进入
protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
}
这里会判断nThreads是否为0,如果在创建NioEventLoop时,不传递大小,默认会创建 cpu个数*2个数量. 继续进入
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
EventExecutorChooserFactory chooserFactory, Object... args) {
//...省略一些代码
if (executor == null) {
//1.创建ThreadPerTaskThread
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
}
children = new EventExecutor[nThreads];
for (int i = 0; i < nThreads; i ++) {
boolean success = false;
try {
//2.创建NioEventLoop
children[i] = newChild(executor, args);
success = true;
} catch (Exception e) {
} finally {
if (!success) {
for (int j = 0; j < i; j ++) {
children[j].shutdownGracefully();
}
for (int j = 0; j < i; j ++) {
EventExecutor e = children[j];
try {
while (!e.isTerminated()) {
e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
}
} catch (InterruptedException interrupted) {
// Let the caller handle the interruption.
Thread.currentThread().interrupt();
break;
}
}
}
}
}
// 3.创建线程选择器
chooser = chooserFactory.newChooser(children);
final FutureListener<Object> terminationListener = new FutureListener<Object>() {
@Override
public void operationComplete(Future<Object> future) throws Exception {
if (terminatedChildren.incrementAndGet() == children.length) {
terminationFuture.setSuccess(null);
}
}
};
for (EventExecutor e: children) {
e.terminationFuture().addListener(terminationListener);
}
Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length);
Collections.addAll(childrenSet, children);
readonlyChildren = Collections.unmodifiableSet(childrenSet);
}
上面代码主要干三件是
2.1. ThreadPerTaskThread
executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
2.2. 创建NioEventLoop线程
首先根据数据大小,创建NioEventLoop队列
children = new EventExecutor[nThreads];
接着为队列中每一个元素创建一个NioEventLoop
children[i] = newChild(executor, args);
会进入
@Override
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
return new NioEventLoop(this, executor, (SelectorProvider) args[0],
((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]);
}
接着进入NioEventLoop的构造方法
NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider,
SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) {
//1.创建TaskQueue
super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler);
if (selectorProvider == null) {
throw new NullPointerException("selectorProvider");
}
if (strategy == null) {
throw new NullPointerException("selectStrategy");
}
provider = selectorProvider;
//2.创建selector
selector = openSelector();
selectStrategy = strategy;
}
这里会做几件事情
2.2.1.创建TaskQueue
protected Queue<Runnable> newTaskQueue(int maxPendingTasks) {
return new LinkedBlockingQueue<Runnable>(maxPendingTasks);
}
就是创建一个阻塞队列
2.2.2.创建selector
调用jdk底层
selector = provider.openSelector();
后面会有一个优化,把jdk原生的selectedKeys,是一个List,改为一个组数,因为数组的查找是O(1),这个感兴趣的可以自己看一下,比较简单
2.3. 创建线程选择器
chooser = chooserFactory.newChooser(children);
点进去之后,会进入
@Override
public EventExecutorChooser newChooser(EventExecutor[] executors) {
if (isPowerOfTwo(executors.length)) {
return new PowerOfTowEventExecutorChooser(executors);
} else {
return new GenericEventExecutorChooser(executors);
}
}
上图可以看到,如果数组队列大小是2的幂次方,就创建PowerOfTowEventExecutorChooser,否则,GenericEventExecutorChooser选择器
这里线程选择器也是一个Netty的优化,比如我们的workGroup的大小是4,当bossGroup监测到一个新的连接后,会把这个新连接放入workGroup,而workGroup里面有四个对象,怎么从这四个对象,就要使用线程选择器,简要概括
- PowerOfTowEventExecutorChooser
使用&运算法,轮训获取下一个.[与运算效率更高]
@Override
public EventExecutor next() {
return executors[idx.getAndIncrement() & executors.length - 1];
}
- GenericEventExecutorChooser
使用 索引+1 ,在对数组大小取模,求出下一个对象
@Override
public EventExecutor next() {
return executors[Math.abs(idx.getAndIncrement() % executors.length)];
}
三. NioEventLoop的启动
到这里,NioEventLoop已经创建好了,那他是什么时候启动的呢?
答案:NioEventLoop第一次添加任务的时候,会判断如果是由外部线程调用的话,就开启这个NioEventLoop
启动的入口
ChannelFuture channelFuture = serverBootstrap.bind(8888).sync();
在doBind()方法中,创建和初始化channel在上一遍文章中已经介绍了源码2
紧着进入doBind0()方法中
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}
上图可以看到,在一个NioEventLoop在添加任务[执行execute(xxx)]方法时候,会判断,如果是外部线程添加的,就开启这个线程,继续进入 channel.eventLoop().execute(xxx)
[SingleThreadEventExecutor]
@Override
public void execute(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
// 1.判断是否在EventLoop线程
boolean inEventLoop = inEventLoop();
if (inEventLoop) {
addTask(task);
} else {
// 2.启动eventLoop线程
startThread();
//3.把这个任务添加到eventLoop线程的taskqueue中
addTask(task);
if (isShutdown() && removeTask(task)) {
reject();
}
}
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}
分析上图三个步骤
3.1 inEventLoop()
这个方法比较简单,就是判断当前线程和eventLoop中的线程是否是同一个, 因为我们启动的是main线程,所以返回是false,说明eventLoop中的线程还没有启动
3.2 startThread(task)
就是启动这个线程
3.3 addTask();
将这个任务包装下,放入eventLoop中的taskQueue中,这个后面会详细讲解
四. NioEventLoop执行
通过上面的流程,NioEventLoop创建好,并且绑定了一个selector,#netty源码分析(二):服务端启动中,我们已经知道这个selector已经注册了服务端的NioServerSocketChannel,随着NioEventLoop的启动,接下来就是要:监听连接事件-->处理连接事件-->执行线程任务了 代码都在 io.netty.channel.nio.NioEventLoop#run()方法中
protected void run() {
for (;;) {
try {
continue;
case SelectStrategy.SELECT:
// 1.调用select方法,获取新的连接
select(wakenUp.getAndSet(false));
if (wakenUp.get()) {
selector.wakeup();
}
default:
// fallthrough
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
// 2.处理IO事件
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
// 3.处理线程池任务
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
逻辑如图
下面我们具体讲解
4.1. 检测IO事件
调用select()方法
private void select(boolean oldWakenUp) throws IOException {
Selector selector = this.selector;
try {
int selectCnt = 0;
long currentTimeNanos = System.nanoTime();
long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
for (;;) {
long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
if (timeoutMillis <= 0) {
if (selectCnt == 0) {
selector.selectNow();
selectCnt = 1;
}
break;
}
// If a task was submitted when wakenUp value was true, the task didn't get a chance to call
// Selector#wakeup. So we need to check task queue again before executing select operation.
// If we don't, the task might be pended until select operation was timed out.
// It might be pended until idle timeout if IdleStateHandler existed in pipeline.
if (hasTasks() && wakenUp.compareAndSet(false, true)) {
selector.selectNow();
selectCnt = 1;
break;
}
int selectedKeys = selector.select(timeoutMillis);
selectCnt ++;
if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks() || hasScheduledTasks()) {
// - Selected something,
// - waken up by user, or
// - the task queue has a pending task.
// - a scheduled task is ready for processing
break;
}
if (Thread.interrupted()) {
// Thread was interrupted so reset selected keys and break so we not run into a busy loop.
// As this is most likely a bug in the handler of the user or it's client library we will
// also log it.
//
// See https://github.com/netty/netty/issues/2426
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely because " +
"Thread.currentThread().interrupt() was called. Use " +
"NioEventLoop.shutdownGracefully() to shutdown the NioEventLoop.");
}
selectCnt = 1;
break;
}
long time = System.nanoTime();
if (time - TimeUnit.MILLISECONDS.toNanos(timeoutMillis) >= currentTimeNanos) {
// timeoutMillis elapsed without anything selected.
selectCnt = 1;
} else if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
// The selector returned prematurely many times in a row.
// Rebuild the selector to work around the problem.
logger.warn(
"Selector.select() returned prematurely {} times in a row; rebuilding Selector {}.",
selectCnt, selector);
rebuildSelector();
selector = this.selector;
// Select again to populate selectedKeys.
selector.selectNow();
selectCnt = 1;
break;
}
currentTimeNanos = time;
}
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
}
} catch (CancelledKeyException e) {
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
selector, e);
}
// Harmless exception - log anyway
}
}
这里netty对selector空轮询进行了处理, 空轮询bug后面会在写一篇文章详细讲解
4.2.处理IO事件
selector.select()监测到IO事件后,就要进行处理了,代码是
processSelectedKeys();
紧着进入
private void processSelectedKeys() {
if (selectedKeys != null) {
processSelectedKeysOptimized(selectedKeys.flip());
} else {
processSelectedKeysPlain(selector.selectedKeys());
}
}
接着接入 NioEventLoop#processSelectedKeysOptimized方法
unsafe类是处理channel中的数据,主要包括数据到读写,在连接过程中,会把新的连接读取到,选择workGroup中的一个EventLoop进行注册到selector上,该selector后续可以处理这个客户端的读写请求,后面会在写文章详细讲解 unsafe类
4.3. reactor线程任务的执行
处理完IO事件后,就要处理线程任务了,根据用户设置的IoRatio参数,设置"处理IO事件"和"执行线程"的时间比例,IoRatio可以在NioEventlLoop设置,如果不设置,默认是50
NioEventLoopGroup bossGroup = new NioEventLoopGroup(1);
bossGroup.setIoRatio(20);
首先根据ioRate比例,算出来执行线程任务要消耗多少时间,然后执行线程任务
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
//ioTime * (100 - ioRatio) / ioRatio 就就是算出时间
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
接着执行线程任务
protected boolean runAllTasks(long timeoutNanos) {
fetchFromScheduledTaskQueue();
Runnable task = pollTask();
if (task == null) {
afterRunningAllTasks();
return false;
}
final long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
long runTasks = 0;
long lastExecutionTime;
for (;;) {
safeExecute(task);
runTasks ++;
// Check timeout every 64 tasks because nanoTime() is relatively expensive.
// XXX: Hard-coded value - will make it configurable if it is really a problem.
if ((runTasks & 0x3F) == 0) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
if (lastExecutionTime >= deadline) {
break;
}
}
task = pollTask();
if (task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
afterRunningAllTasks();
this.lastExecutionTime = lastExecutionTime;
return true;
}
上图核心逻辑,就是
- 1.从taskqueue中取出任务
- 2.执行线程 [safeExecute(task)],注意这里是执行调用线程的run方法,是串行阻塞调用
- 3.继续从taskqueue中取出下一个任务执行,执行64次个任务后,判断一下时间是否过期,过期的话,就停止执行
五. NioEventLoop总结
NioEventLoop是一个线程,里面有一个selector,服务端的selector关注的是连接请求,当有新的连接接入时候,会创建NioSocketChannel并注册到workGroup的一个NioEventLoop上,NioEventLoop启动后,会循环做三件事情:
- 检测IO事件
- 处理IO事件
- 执行线程任务
