Netty源码之启动服务
Netty本身是基于JDK的NIO进行优化和改进的框架，本身还是基于JDK的NIO实现，所以再瞅一眼NIO：
从上图可以看到，要启动NIO，那么就需要创建Selector，同样Netty要启动，同样也需要创建Selector。
那么我们可以采用调试跟踪的方式来分析源码。
主线:
主线程
- 创建Selector
- 创建ServerSocketChannel
- 初始化ServerSocketChannel
- 给ServerSocketChannel 从 boss group 中选择一个 NioEventLoop
BossThread
- 将ServerSocketChannel 注册到选择的 NioEventLoop的Selector
- 绑定地址启动
- 注册接受连接事件（OP_ACCEPT）到Selector 上
主线程
依旧从example中的EchoServer看起
创建Selector
上篇文章写过，NioEventLoopGroup的构造方法中使用了JDK的SelectorProvider.provider()来创建selector。
创建ServerSocketChannel
ServerSocketChannel再bind方法中创建，进入doBind方法后：
final ChannelFuture regFuture = initAndRegister();
再进入initAndRegister：
channel = channelFactory.newChannel();
初始化ServerSocketChannel
还是initAndRegister进入init方法，由于channel为ServerSocketChannel，所以进入的是ServerBootstrap：
void init(Channel channel) {
//参数和属性设置
setChannelOptions(channel, options0().entrySet().toArray(newOptionArray(0)), logger);
setAttributes(channel, attrs0().entrySet().toArray(newAttrArray(0)));
ChannelPipeline p = channel.pipeline();
final EventLoopGroup currentChildGroup = childGroup;
final ChannelHandler currentChildHandler = childHandler;
final Entry<ChannelOption<?>, Object>[] currentChildOptions =
childOptions.entrySet().toArray(newOptionArray(0));
final Entry<AttributeKey<?>, Object>[] currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(0));
// 这里需要了解责任链模式pipeline
// ChannelInitializer一次性、初始化handler:
// 负责添加一个ServerBootstrapAcceptor handler，添加完后，自己就移除了:
// ServerBootstrapAcceptor handler： 负责接收客户端连接创建连接后，对连接的初始化工作。
p.addLast(new ChannelInitializer<Channel>() {
@Override
public void initChannel(final Channel ch) {
final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
pipeline.addLast(handler);
}
ch.eventLoop().execute(new Runnable() {
@Override
public void run() {
pipeline.addLast(new ServerBootstrapAcceptor(
ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
}
});
}
});
}
给ServerSocketChannel从bossGroup中选择一个NioEventLoop
还是initAndRegister方法：
// 这里是返回一个Future，走的是异步过程
ChannelFuture regFuture = config().group().register(channel);
if (regFuture.cause() != null) {
if (channel.isRegistered()) {
channel.close();
} else {
channel.unsafe().closeForcibly();
}
}
进入MultithreadEventLoopGroup的register方法：
@Override
public ChannelFuture register(Channel channel) {
return next().register(channel);
}
这里的next()返回的是SingleThreadEventLoop，进入register方法：
@Override
public ChannelFuture register(final ChannelPromise promise) {
ObjectUtil.checkNotNull(promise, "promise");
promise.channel().unsafe().register(this, promise);
return promise;
}
依旧进入register，对应类为AbstractUnsafe：
@Override
public final void register(EventLoop eventLoop, final ChannelPromise promise) {
...
if (eventLoop.inEventLoop()) {
// 判断是不是eventLoop线程（Netty中很多这种用法）
register0(promise);
} else {
try {
// 如果不是eventLoop线程，则使用eventLoop来执行
eventLoop.execute(new Runnable() {
@Override
public void run() {
// NioeventLoop的执行
register0(promise);
}
});
} catch (Throwable t) {
...
}
}
}
最终到register0方法进行处理。
private void register0(ChannelPromise promise) {
try {
...
boolean firstRegistration = neverRegistered;
// jdk底层操作，channel注册到selector上
doRegister();
neverRegistered = false;
// register完成
registered = true;
// Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
// user may already fire events through the pipeline in the ChannelFutureListener.
// 前面init方法将部分handler加入到了pipeline，这里做了一次确认
pipeline.invokeHandlerAddedIfNeeded();
// 通知提交register操作的线程
safeSetSuccess(promise);
// 通知所有关心register时间的handler
pipeline.fireChannelRegistered();
// Only fire a channelActive if the channel has never been registered. This prevents firing
// multiple channel actives if the channel is deregistered and re-registered.
// server socket的注册不会走进下面if，server socket接受连接创建的socket可以走进去。因为accept后就active了。
if (isActive()) {
if (firstRegistration) {
pipeline.fireChannelActive();
} else if (config().isAutoRead()) {
// This channel was registered before and autoRead() is set. This means we need to begin read
// again so that we process inbound data.
//
// See https://github.com/netty/netty/issues/4805
// 让channel监听OP_READ_事件
beginRead();
}
}
} catch (Throwable t) {
...
}
}
BossThread
将ServerSocketChannel注册到选择的NioEventLoop的Selector
从上面register0方法中进入doRegister方法，对应的类为AbstractNioChannel：
selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
绑定地址启动
回到AbstractBootstrap的doBind方法，进入doBind0方法：
private static void doBind0(
final ChannelFuture regFuture, final Channel channel,
final SocketAddress localAddress, final ChannelPromise promise) {
// This method is invoked before channelRegistered() is triggered. Give user handlers a chance to set up
// the pipeline in its channelRegistered() implementation.
channel.eventLoop().execute(new Runnable() {
@Override
public void run() {
if (regFuture.isSuccess()) {
channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
} else {
promise.setFailure(regFuture.cause());
}
}
});
}
再进入bind方法，此时进入的是AbstractUnsafe的bind方法：
@Override
public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
...
boolean wasActive = isActive();
try {
// NioServerSocketChannel
doBind(localAddress);
} catch (Throwable t) {
safeSetFailure(promise, t);
closeIfClosed();
return;
}
// 绑定后，才开始激活
if (!wasActive && isActive()) {
invokeLater(new Runnable() {
@Override
public void run() {
pipeline.fireChannelActive();
}
});
}
safeSetSuccess(promise);
}
进入NioServerSocketChannel的doBind方法：
@Override
protected void doBind(SocketAddress localAddress) throws Exception {
if (PlatformDependent.javaVersion() >= 7) {
javaChannel().bind(localAddress, config.getBacklog());
} else {
javaChannel().socket().bind(localAddress, config.getBacklog());
}
}
此处就是进行地址绑定启动的地方。
注册接受连接事件（OP_ACCEPT）到Selector上
回到AbstractUnsafe的bind方法有这么一段：
// 绑定后，才开始激活
if (!wasActive && isActive()) {
invokeLater(new Runnable() {
@Override
public void run() {
pipeline.fireChannelActive();
}
});
}
进入pipeline.fireChannelActive()方法，这里有几次方法的进入，过程如下：
DefaultChannelPipeline：
public final ChannelPipeline fireChannelActive() {
AbstractChannelHandlerContext.invokeChannelActive(head);
return this;
}
进入invokeChannelActive
static void invokeChannelActive(final AbstractChannelHandlerContext next) {
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
next.invokeChannelActive();
} else {
executor.execute(new Runnable() {
@Override
public void run() {
next.invokeChannelActive();
}
});
}
}
进入invokeChannelActive
private void invokeChannelActive() {
if (invokeHandler()) {
try {
((ChannelInboundHandler) handler()).channelActive(this);
} catch (Throwable t) {
notifyHandlerException(t);
}
} else {
fireChannelActive();
}
}
进入channelActive，此时依旧在DefaultChannelPipeline类中：
public void channelActive(ChannelHandlerContext ctx) {
ctx.fireChannelActive();
// 注册读事件：读包括：创建连接/读数据）
readIfIsAutoRead();
}
进入readIfIsAutoRead：
private void readIfIsAutoRead() {
if (channel.config().isAutoRead()) {
channel.read();
}
}
进入read：
public void read(ChannelHandlerContext ctx) {
// 创建连接或者读事件，即注册OP_ACCEPT/OP_READ事件
unsafe.beginRead();
}
进入beginRead的doBeginRead方法，在AbstractNioChannel类中：
protected void doBeginRead() throws Exception {
// Channel.read() or ChannelHandlerContext.read() was called
final SelectionKey selectionKey = this.selectionKey;
if (!selectionKey.isValid()) {
return;
}
readPending = true;
final int interestOps = selectionKey.interestOps();
// 假设之前没有监听readInterestOp，则监听readInterestOp
if ((interestOps & readInterestOp) == 0) {
// NioServerSocketChannel: readInterestOp = OP_ACCEPT = 1 << 4 = 16
logger.info("interest ops： " + readInterestOp);
selectionKey.interestOps(interestOps | readInterestOp);
}
}
Netty的责任链设计
前面对Netty的使用中可以看出，不需要自己创建pipeline，因为使用ServerBootstrap或者Bootstrap启动服务端或者客户端时，Netty会为每个Channel连接创建一个独立的pipeline。对于使用者而言，只需要将自定义的拦截器加入到pipeline中即可。
ChannelPipeline支持运行态动态的添加或者删除ChannelHandler，在某些场景下这个特性非常实用。例如当业务高峰期需要对系统做拥塞保护时，就可以根据当前的系统时间进行判断，如果处于业务高峰期，则动态地将系统拥塞保护ChannelHandler添加到当前的ChannelPipeline中，当高峰期过去之后，就可以动态删除拥塞保护ChannelHandler了。
ChannelPipeline是线程安全的,这意味着N个业务线程可以并发地操作ChannelPipeline而不存在多线程并发问题。但是，ChannelHandler却不是线程安全的，这意味着尽管ChannelPipeline是线程安全的，但是用户仍然需要自己保证ChannelHandler的线程安全。
netty的责任处理器接口
ChannelHandler为pipeline中的所有的handler的顶级抽象接口，它规定了所有的handler统一要有添加、移除、异常捕获的行为。netty对责任处理接口，做了更细粒度的划分, 处理器被分成了两种, 一种是入站处理器 ChannelInboundHandler，另一种是出站处理器 ChannelOutboundHandler。
public interface ChannelHandler {
/**
* Gets called after the {@link ChannelHandler} was added to the actual context and it's ready to handle events.
*/
void handlerAdded(ChannelHandlerContext ctx) throws Exception;
/**
* Gets called after the {@link ChannelHandler} was removed from the actual context and it doesn't handle events
* anymore.
*/
void handlerRemoved(ChannelHandlerContext ctx) throws Exception;
/**
* Gets called if a {@link Throwable} was thrown.
*
* @deprecated if you want to handle this event you should implement {@link ChannelInboundHandler} and
* implement the method there.
*/
@Deprecated
void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception;
...
}
ChannelInboundHandler中的方法将会在数据被接收时或者与其对应的Channel状态发生改变时被调用。这些方法和Channel的生命周期密切相关。
public interface ChannelInboundHandler extends ChannelHandler {
/**
* 当Channel 已经注册到它的EventLoop 并且能够处理I/O 时被调用
*/
void channelRegistered(ChannelHandlerContext ctx) throws Exception;
/**
* 当Channel 从它的EventLoop 注销并且无法处理任何I/O 时被调用
*/
void channelUnregistered(ChannelHandlerContext ctx) throws Exception;
/**
* 当Channel 处于活动状态时被调用；Channel 已经连接/绑定并且已经就绪
*/
void channelActive(ChannelHandlerContext ctx) throws Exception;
/**
* 当Channel 离开活动状态并且不再连接它的远程节点时被调用
*/
void channelInactive(ChannelHandlerContext ctx) throws Exception;
/**
* 当从Channel 读取数据时被调用
*/
void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception;
/**
* 当Channel 离开活动状态并且不再连接它的远程节点时被调用
*/
void channelReadComplete(ChannelHandlerContext ctx) throws Exception;
/**
* 当ChannelInboundHandler.fireUserEventTriggered()方法被调用时被调用。
*/
void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception;
/**
* 当Channel 的可写状态发生改变时被调用。可以通过调用Channel 的isWritable()方法来检测Channel 的可写性。与可写性相关的阈值可以通过Channel.config().setWriteHighWaterMark()和Channel.config().setWriteLowWaterMark()方法来设置
*/
void channelWritabilityChanged(ChannelHandlerContext ctx) throws Exception;
/**
* Gets called if a {@link Throwable} was thrown.
*/
@Override
@SuppressWarnings("deprecation")
void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception;
}
ChannelOutboundHandler处理出站操作和数据。它的方法将被Channel、Channel-Pipeline以及ChannelHandlerContext调用。
public interface ChannelOutboundHandler extends ChannelHandler {
/**
* 当请求将Channel 绑定到本地地址时被调用
*/
void bind(ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise) throws Exception;
/**
* 当请求将Channel 连接到远程节点时被调用
*/
void connect(
ChannelHandlerContext ctx, SocketAddress remoteAddress,
SocketAddress localAddress, ChannelPromise promise) throws Exception;
/**
* 当请求将Channel 从远程节点断开时被调用
*/
void disconnect(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception;
/**
* 当请求关闭Channel 时被调用
*/
void close(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception;
/**
* 当请求将Channel 从它的EventLoop 注销时被调用
*/
void deregister(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception;
/**
* 当请求从Channel 读取更多的数据时被调用
*/
void read(ChannelHandlerContext ctx) throws Exception;
/**
* 当请求通过Channel 将数据写到远程节点时被调用
*/
void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception;
/**
* 当请求通过Channel 将入队数据冲刷到远程节点时被调用
*/
void flush(ChannelHandlerContext ctx) throws Exception;
}
添加删除责任处理器的接口
netty中所有的处理器最终都在添加在pipeline上，所以添加删除责任处理器的接口的行为 netty在channelPipeline中的进行了规定。
这里不贴源码了，直接贴一段channelPipeline的注释：
+---------------------------------------------------+---------------+
| ChannelPipeline | |
| \|/ |
| +---------------------+ +-----------+----------+ |
| | Inbound Handler N | | Outbound Handler 1 | |
| +----------+----------+ +-----------+----------+ |
| /|\ | |
| | \|/ |
| +----------+----------+ +-----------+----------+ |
| | Inbound Handler N-1 | | Outbound Handler 2 | |
| +----------+----------+ +-----------+----------+ |
| /|\ . |
| . . |
| ChannelHandlerContext.fireIN_EVT() ChannelHandlerContext.OUT_EVT()|
| [ method call] [method call] |
| . . |
| . \|/ |
| +----------+----------+ +-----------+----------+ |
| | Inbound Handler 2 | | Outbound Handler M-1 | |
| +----------+----------+ +-----------+----------+ |
| /|\ | |
| | \|/ |
| +----------+----------+ +-----------+----------+ |
| | Inbound Handler 1 | | Outbound Handler M | |
| +----------+----------+ +-----------+----------+ |
| /|\ | |
+---------------+-----------------------------------+---------------+
| \|/
+---------------+-----------------------------------+---------------+
| | | |
| [ Socket.read() ] [ Socket.write() ] |
| |
| Netty Internal I/O Threads (Transport Implementation) |
+-------------------------------------------------------------------+
上下文
pipeline中的handler被封装进了上下文ChannelHandlerContext。通过上下文拿到当前节点所属的channel, 以及它的线程执行器。
/**
* 获取ChannelHandlerContext所对应的这个Channel对象
*/
Channel channel();
/**
* 获取事件执行器
*/
EventExecutor executor();
ChannelHandlerContext继承自ChannelInboundInvoker和ChannelOutboundInvoker方法描述如下：
| 字段 | 描述 | |
|---|---|---|
| alloc | 返回和这个实例相关联的Channel 所配置的ByteBufAllocator | |
| bind | 绑定到给定的SocketAddress，并返回ChannelFuture | |
| channel | 返回绑定到这个实例的Channel | |
| close | 关闭Channel，并返回ChannelFuture | |
| connect | 连接给定的SocketAddress，并返回ChannelFuture | |
| deregister | 从之前分配的EventExecutor 注销，并返回ChannelFuture | |
| disconnect | 从远程节点断开，并返回ChannelFuture | |
| executor | 返回调度事件的EventExecutor | |
| fireChannelActive | 触发对下一个ChannelInboundHandler 上的channelActive()方法（已连接）的调用 | |
| fireChannelInactive | 触发对下一个ChannelInboundHandler 上的channelInactive()方法（已关闭）的调用 | |
| fireChannelRead | 触发对下一个ChannelInboundHandler 上的channelRead()方法（已接收的消息）的调用 | |
| fireChannelReadComplete | 触发对下一个ChannelInboundHandler 上的channelReadComplete()方法的调用 | |
| fireChannelRegistered | 触发对下一个ChannelInboundHandler 上的fireChannelRegistered()方法的调用 | |
| fireChannelUnregistered | 触发对下一个ChannelInboundHandler 上的fireChannelUnregistered()方法的调用 | |
| fireChannelWritabilityChanged | 触发对下一个ChannelInboundHandler 上的fireChannelWritabilityChanged()方法的调用 | |
| fireExceptionCaught | 触发对下一个ChannelInboundHandler 上的fireExceptionCaught(Throwable)方法的调用 | |
| fireUserEventTriggered | 触发对下一个ChannelInboundHandler 上的fireUserEventTriggered(Object evt)方法的调用 | |
| handler | 返回绑定到这个实例的ChannelHandler | |
| isRemoved | 如果所关联的ChannelHandler 已经被从ChannelPipeline中移除则返回true | |
| name | 返回这个实例的唯一名称 | |
| pipeline | 返回这个实例所关联的ChannelPipeline | |
| read | 将数据从Channel读取到第一个入站缓冲区；如果读取成功则触发一个channelRead事件，并（在最后一个消息被读取完成后）通知ChannelInboundHandler 的channelReadComplete(ctx)方法 | |
| write | 通过这个实例写入消息并经过ChannelPipeline | |
| writeAndFlush | 通过这个实例写入并冲刷消息并经过ChannelPipeline | 将数据从Channel读取到第一个入站缓冲区；如果读取成功则触发一个channelRead事件，并（在最后一个消息被读取完成后）通知ChannelInboundHandler 的channelReadComplete(ctx)方法 |
| write | 通过这个实例写入消息并经过ChannelPipeline | |
| writeAndFlush | 通过这个实例写入并冲刷消息并经过ChannelPipeline |
责任终止机制
- 在pipeline中的任意一个节点，只要我们不手动的往下传播下去，这个事件就会终止传播在当前节点；
- 对于入站数据，默认会传递到尾节点进行回收。如果我们不进行下一步传播，事件就会终止在当前节点，别忘记回收msg；
- 对于出站数据，用header节点的使用unsafe对象，把数据写会客户端也意味着事件的终止。
事件的传播
底层事件的传播使用的就是链表
private AbstractChannelHandlerContext findContextOutbound(int mask) {
AbstractChannelHandlerContext ctx = this;
do {
ctx = ctx.prev;
} while ((ctx.executionMask & mask) == 0);
return ctx;
}
Netty的责任链源码解读
看到EchoServer中添加了一个LoggingHandler，进入该类且以channelRegistered为例：
public void channelRegistered(ChannelHandlerContext ctx) throws Exception {
if (logger.isEnabled(internalLevel)) {
logger.log(internalLevel, format(ctx, "REGISTERED"));
}
ctx.fireChannelRegistered();
}
进入AbstractChannelHandlerContext的fireChannelRegistered：
public ChannelHandlerContext fireChannelRegistered() {
invokeChannelRegistered(findContextInbound(MASK_CHANNEL_REGISTERED));
return this;
}
再往invokeChannelRegistered中看就是获取next channelHandler去运行channelRegistered方法。
Netty构建连接
从前面知道无论绑定IP、处理连接都是NioEventLoop来处理，于是看一下对应的run方法，进入processSelectedKeys方法：
private void processSelectedKeys() {
if (selectedKeys != null) {
// 不用JDK的selector.selectedKeys()，性能更好（1%-2%），垃圾回收更少
processSelectedKeysOptimized();
} else {
// JDK的selector.selectedKeys()
processSelectedKeysPlain(selector.selectedKeys());
}
}
看下netty如何优化，进入processSelectedKeysOptimized：
private void processSelectedKeysOptimized() {
for (int i = 0; i < selectedKeys.size; ++i) {
final SelectionKey k = selectedKeys.keys[i];
// See https://github.com/netty/netty/issues/2363
// 清空数组元素，在通道关闭后对其进行GC
selectedKeys.keys[i] = null;
// AbstractNioChannel中的doRegister()方法中
// selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
final Object a = k.attachment();
// NioSocketChannel
if (a instanceof AbstractNioChannel) {
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
...
}
}
进入第一个processSelectedKey看到这一段：
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
// 处理读请求（断开连接）或接入连接
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
这里ops的标志代表着什么？
OP_READ:INT = 1 << 0
OP_WRITE:INT = 1 << 2
OP_CONNECT:INT = 1 << 3
OP_ACCEPT:INT = 1 << 4
现在启动一个客户端连接，再回到int readyOps = k.readyOps()，目前ops=16，表示OP_ACCEPT，所以代码进入这里：
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
进入AbstractNioMessageChannel的read方法，再进入NioServerSocketChannel的doReadMessages方法：
protected int doReadMessages(List<Object> buf) throws Exception {
// 接受新连接创建SocketChannel
SocketChannel ch = SocketUtils.accept(javaChannel());
try {
if (ch != null) {
buf.add(new NioSocketChannel(this, ch));
return 1;
}
} catch (Throwable t) {
logger.warn("Failed to create a new channel from an accepted socket.", t);
try {
ch.close();
} catch (Throwable t2) {
logger.warn("Failed to close a socket.", t2);
}
}
return 0;
}
进入accept方法
public static SocketChannel accept(final ServerSocketChannel serverSocketChannel) throws IOException {
try {
return AccessController.doPrivileged(new PrivilegedExceptionAction<SocketChannel>() {
@Override
public SocketChannel run() throws IOException {
// 非阻塞模式下，没有连接请求时，返回null
return serverSocketChannel.accept();
}
});
} catch (PrivilegedActionException e) {
throw (IOException) e.getCause();
}
}
整体流程便结束了
Netty接收数据
进入NioEventLoop的processSelectedKey方法看这段：
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
第一次进入时这里是NioServerSocketChannel说明走的是创建连接，而第二次进入时这里是NioSocketChannel，是读事件。如果打断点的话会发现两次进入的方法除了对象不同，线程也不同，因为这里采用了主从的方式。且read方法对应的是AbstractNioByteChannel类。
读数据技巧
AbstractNioByteChannel的read方法：
public final void read() {
final ChannelConfig config = config();
if (shouldBreakReadReady(config)) {
clearReadPending();
return;
}
final ChannelPipeline pipeline = pipeline();
// Buffer的分配器
final ByteBufAllocator allocator = config.getAllocator();
// io.netty.channel.DefaultChannelConfig中设置RecvByteBufAllocator，默认AdaptiveRecvByteBufAllocator
final RecvByteBufAllocator.Handle allocHandle = recvBufAllocHandle();
allocHandle.reset(config);
ByteBuf byteBuf = null;
boolean close = false;
try {
do {
// 尽可能分配合适的大小：猜
byteBuf = allocHandle.allocate(allocator);
// 读并且记录读了多少，如果读满了，下次continue的话就直接扩容。
allocHandle.lastBytesRead(doReadBytes(byteBuf));
if (allocHandle.lastBytesRead() <= 0) {
// nothing was read. release the buffer.
byteBuf.release();
byteBuf = null;
close = allocHandle.lastBytesRead() < 0;
if (close) {
// There is nothing left to read as we received an EOF.
readPending = false;
}
break;
}
allocHandle.incMessagesRead(1);
readPending = false;
// pipeline上执行，业务逻辑的处理就在这个地方
pipeline.fireChannelRead(byteBuf);
byteBuf = null;
} while (allocHandle.continueReading());
// 记录这次读事件总共读了多少数据，计算下次分配大小。
allocHandle.readComplete();
// 相当于完成本次读事件的处理
pipeline.fireChannelReadComplete();
if (close) {
closeOnRead(pipeline);
}
} catch (Throwable t) {
handleReadException(pipeline, byteBuf, t, close, allocHandle);
} finally {
// Check if there is a readPending which was not processed yet.
// This could be for two reasons:
// * The user called Channel.read() or ChannelHandlerContext.read() in channelRead(...) method
// * The user called Channel.read() or ChannelHandlerContext.read() in channelReadComplete(...) method
//
// See https://github.com/netty/netty/issues/2254
if (!readPending && !config.isAutoRead()) {
removeReadOp();
}
}
}
每次该读多大
自适应数据大小的分配器（AdaptiveRecvByteBufAllocator）
第一次靠猜，默认是1024
public ByteBuf allocate(ByteBufAllocator alloc) {
return alloc.ioBuffer(guess());
}
public int guess() {
return nextReceiveBufferSize;
}
然后会根据情况进行buffer大小的调整，默认都是这个AdaptiveRecvByteBufAllocator的record方法来调整。
/**
* 接受数据buffer的容量会尽可能的足够大以接受数据
* 也尽可能的小以不会浪费它的空间
* @param actualReadBytes
*/
private void record(int actualReadBytes) {
// 尝试是否可以减小分配的空间仍然能满足需求：
// 尝试方法：当前实际读取的size是否小于或等于打算缩小的尺寸
if (actualReadBytes <= SIZE_TABLE[max(0, index - INDEX_DECREMENT - 1)]) {
// decreaseNow: 连续2次尝试减小都可以
if (decreaseNow) {
//减小
index = max(index - INDEX_DECREMENT, minIndex);
nextReceiveBufferSize = SIZE_TABLE[index];
decreaseNow = false;
} else {
decreaseNow = true;
}
// 判断是否实际读取的数据大于等于预估的，如果是，尝试扩容
} else if (actualReadBytes >= nextReceiveBufferSize) {
index = min(index + INDEX_INCREMENT, maxIndex);
nextReceiveBufferSize = SIZE_TABLE[index];
decreaseNow = false;
}
}
不过这里有两种情况，第一种就是读的数据超过1024，那么就要扩容，如果是小于1024，那么就要缩容。这里看lastBytesRead方法：
public void lastBytesRead(int bytes) {
// If we read as much as we asked for we should check if we need to ramp up the size of our next guess.
// This helps adjust more quickly when large amounts of data is pending and can avoid going back to
// the selector to check for more data. Going back to the selector can add significant latency for large
// data transfers.
// 判断是否预估的空间都被“读”到的数据填满了，如果填满了，尝试扩容再试试。
if (bytes == attemptedBytesRead()) {
record(bytes);
}
super.lastBytesRead(bytes);
}
还是进入record方法
private void record(int actualReadBytes) {
// 尝试是否可以减小分配的空间仍然能满足需求：
// 尝试方法：当前实际读取的size是否小于或等于打算缩小的尺寸
if (actualReadBytes <= SIZE_TABLE[max(0, index - INDEX_DECREMENT - 1)]) {
// decreaseNow: 连续2次尝试减小都可以
if (decreaseNow) {
//减小
index = max(index - INDEX_DECREMENT, minIndex);
nextReceiveBufferSize = SIZE_TABLE[index];
decreaseNow = false;
} else {
decreaseNow = true;
}
// 判断是否实际读取的数据大于等于预估的，如果是，尝试扩容
} else if (actualReadBytes >= nextReceiveBufferSize) {
index = min(index + INDEX_INCREMENT, maxIndex);
nextReceiveBufferSize = SIZE_TABLE[index];
decreaseNow = false;
}
}
static {
List<Integer> sizeTable = new ArrayList<Integer>();
// 16、32、48、64、80...496: + 16
// 小于512时，增加64，减小16
for (int i = 16; i < 512; i += 16) {
sizeTable.add(i);
}
// 512、512*2、512*4、512*8、512*16直到整形最大值,
// 大于512时，16倍增大，1倍减小
// 后面会判断最大，最小限制，默认在64和65536之间。
for (int i = 512; i > 0; i <<= 1) {
sizeTable.add(i);
}
SIZE_TABLE = new int[sizeTable.size()];
for (int i = 0; i < SIZE_TABLE.length; i ++) {
SIZE_TABLE[i] = sizeTable.get(i);
}
}
上面代码中需要连续判断两次才会进入这个方法：nextReceiveBufferSize = SIZE_TABLE[index];。这表明了AdaptiveRecvByteBufAllocator对bytebuf的猜测：放大果断，缩小谨慎。
Netty业务处理
AbstractNioByteChannel中的read方法有段do-while循环中有句pipeline.fireChannelRead(byteBuf)，前面介绍过pipeline的运作流程，这里从fireChannelRead进入，跳转几次后进入自己写的业务handler。简单过以下流程：
DefaultChannelPipeline:
@Override
public final ChannelPipeline fireChannelRead(Object msg) {
AbstractChannelHandlerContext.invokeChannelRead(head, msg);
return this;
}
AbstractChannelHandlerContext:
static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
next.invokeChannelRead(m);
} else {
executor.execute(new Runnable() {
@Override
public void run() {
next.invokeChannelRead(m);
}
});
}
}
private void invokeChannelRead(Object msg) {
if (invokeHandler()) {
try {
((ChannelInboundHandler) handler()).channelRead(this, msg);
} catch (Throwable t) {
notifyHandlerException(t);
}
} else {
fireChannelRead(msg);
}
}
LoggingHandler的channelRead方法：
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
if (logger.isEnabled(internalLevel)) {
logger.log(internalLevel, format(ctx, "READ", msg));
}
ctx.fireChannelRead(msg);
}
AbstractChannelHandlerContext：
public ChannelHandlerContext fireChannelRead(final Object msg) {
invokeChannelRead(findContextInbound(MASK_CHANNEL_READ), msg);
return this;
}
private AbstractChannelHandlerContext findContextInbound(int mask) {
AbstractChannelHandlerContext ctx = this;
do {
ctx = ctx.next;
} while ((ctx.executionMask & mask) == 0);
// executionMask是把handler进行添加的时候算出来的值（查看ChannelHandlerMask类中常量），然后判断是否存在
return ctx;
}
此时这里的next为EchoServerHandler，也正如EchoServer中添加handler的顺序，LoggingHandler滞后就是EchoServerHandler。
总结一下就是：数据在pipeline中所有的handler的channelRead()执行过程中，Handler 要实现io.netty.channel.ChannelInboundHandler#channelRead 且不能加注解 @Skip才能被执行到。
Netty写数据
write ：写到一个 buffer
从EchoServerHandler开始
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
ctx.write(msg);
}
进入AbstractChannelHandlerContext的write方法
public ChannelFuture write(final Object msg, final ChannelPromise promise) {
write(msg, false, promise);
return promise;
}
再进入write方法，看到获取next后调用了invokeWriteAndFlush方法，一路看到DefaultChannelPipeline的write方法：
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) {
unsafe.write(msg, promise);
}
然后进入的是AbstractUnsafe的write方法：
public final void write(Object msg, ChannelPromise promise) {
assertEventLoop();
ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
// 下面的判断，是判断是否channel已经关闭了。
if (outboundBuffer == null) {
// If the outboundBuffer is null we know the channel was closed and so
// need to fail the future right away. If it is not null the handling of the rest
// will be done in flush0()
// See https://github.com/netty/netty/issues/2362
safeSetFailure(promise, newClosedChannelException(initialCloseCause));
// release message now to prevent resource-leak
ReferenceCountUtil.release(msg);
return;
}
int size;
try {
msg = filterOutboundMessage(msg);
size = pipeline.estimatorHandle().size(msg);
if (size < 0) {
size = 0;
}
} catch (Throwable t) {
safeSetFailure(promise, t);
ReferenceCountUtil.release(msg);
return;
}
outboundBuffer.addMessage(msg, size, promise);
}
进入addMessage中的incrementPendingOutboundBytes方法：
private void incrementPendingOutboundBytes(long size, boolean invokeLater) {
if (size == 0) {
return;
}
long newWriteBufferSize = TOTAL_PENDING_SIZE_UPDATER.addAndGet(this, size);
// 判断待发送的数据的size是否高于高水位线
if (newWriteBufferSize > channel.config().getWriteBufferHighWaterMark()) {
setUnwritable(invokeLater);
}
}
private void setUnwritable(boolean invokeLater) {
for (;;) {
final int oldValue = unwritable;
final int newValue = oldValue | 1;
// 设置unwritable
if (UNWRITABLE_UPDATER.compareAndSet(this, oldValue, newValue)) {
if (oldValue == 0 && newValue != 0) {
fireChannelWritabilityChanged(invokeLater);
}
break;
}
}
}
Netty待写数据太多，超过一定的水位线channel.config().getWriteBufferHighWaterMark()即writeBufferWaterMark.high()时，会将可写的标志位改成false ，让应用端自己做决定要不要发送数据。该水位为：private static final int DEFAULT_HIGH_WATER_MARK = 64 * 1024;
flush : 把 buffer 里的数据发送出去
接着从EchoServerHandler的channelReadComplete开始看：
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush();
}
AbstractChannelHandlerContext:
public ChannelHandlerContext flush() {
final AbstractChannelHandlerContext next = findContextOutbound(MASK_FLUSH);
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
next.invokeFlush();
} else {
Tasks tasks = next.invokeTasks;
if (tasks == null) {
next.invokeTasks = tasks = new Tasks(next);
}
safeExecute(executor, tasks.invokeFlushTask, channel().voidPromise(), null);
}
return this;
}
跟着next.invokeFlush()一路进入到AbstractUnsafe的flush方法
public final void flush() {
assertEventLoop();
ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
// outboundBuffer == null表明channel关闭了。
if (outboundBuffer == null) {
return;
}
outboundBuffer.addFlush();
flush0();
}
进入flush0中的doWrite方法：
protected void doWrite(ChannelOutboundBuffer in) throws Exception {
SocketChannel ch = javaChannel();
// 有数据要写，且能写入，这最多尝试16次
int writeSpinCount = config().getWriteSpinCount();
do {
if (in.isEmpty()) {
// All written so clear OP_WRITE
// 数据都写完了,不用也不需要写16次
clearOpWrite();
// Directly return here so incompleteWrite(...) is not called.
return;
}
// Ensure the pending writes are made of ByteBufs only.
int maxBytesPerGatheringWrite = ((NioSocketChannelConfig) config).getMaxBytesPerGatheringWrite();
// 最多返回1024个数据，总的size尽量不超过maxBytesPerGatheringWrite
ByteBuffer[] nioBuffers = in.nioBuffers(1024, maxBytesPerGatheringWrite);
int nioBufferCnt = in.nioBufferCount();
// Always us nioBuffers() to workaround data-corruption.
// See https://github.com/netty/netty/issues/2761
// 是否是单个数据，还是批量数据
switch (nioBufferCnt) {
case 0:
// We have something else beside ByteBuffers to write so fallback to normal writes.
writeSpinCount -= doWrite0(in);
break;
case 1: {
// 单个数据
// Only one ByteBuf so use non-gathering write
// Zero length buffers are not added to nioBuffers by ChannelOutboundBuffer, so there is no need
// to check if the total size of all the buffers is non-zero.
ByteBuffer buffer = nioBuffers[0];
int attemptedBytes = buffer.remaining();
final int localWrittenBytes = ch.write(buffer);
if (localWrittenBytes <= 0) {
incompleteWrite(true);
return;
}
adjustMaxBytesPerGatheringWrite(attemptedBytes, localWrittenBytes, maxBytesPerGatheringWrite);
// 从ChannelOutboundBuffer中移除已经写出的数据
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
default: {
// 批量数据
// Zero length buffers are not added to nioBuffers by ChannelOutboundBuffer, so there is no need
// to check if the total size of all the buffers is non-zero.
// We limit the max amount to int above so cast is safe
long attemptedBytes = in.nioBufferSize();
final long localWrittenBytes = ch.write(nioBuffers, 0, nioBufferCnt);
if (localWrittenBytes <= 0) {
//缓存区满了，写不进去了，注册写事件。
incompleteWrite(true);
return;
}
// Casting to int is safe because we limit the total amount of data in the nioBuffers to int above.
adjustMaxBytesPerGatheringWrite((int) attemptedBytes, (int) localWrittenBytes,
maxBytesPerGatheringWrite);
in.removeBytes(localWrittenBytes);
--writeSpinCount;
break;
}
}
} while (writeSpinCount > 0);
// 写了16次数据，还是没有写完，直接schedule一个新的flush task出来。而不是注册写事件。
incompleteWrite(writeSpinCount < 0);
}
这里有两个点：
- 只要有数据要写且能写时，则一直尝试，直到16次（writeSpinCount）。若写16次还没有写完，就直接schedule一个task来继续写，而不是用注册写事件来触发，这样更简洁。
- 批量写数据时，如果尝试写的都写进去了，接下来会尝试写更多（maxBytesPerGatheringWrite）
