Netty之ByteBuf

会灰翔的灰机
98 阅读4分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路。

版本：4.0.35.Final netty的ByteBuf结构还是比较多元化的，文章主要学习他的pooled与unpooled，unsafe与非unsafe的内存结构的区别 了解过JDK ByteBuffer的同学应该知道其主要有两种类型，即HeapByteBuffer、MappedByteBuffer（子类就是DirectByteBuffer）。词如其名，一个堆内存一个直接内存（堆外内存）。同样netty也是有同类的对应实现 image.png

堆内内存

UnpooledHeapByteBuf

可以看到构造器中堆内存的申请其实就是一个new byte数组的构建。对于内存的操作也就是直接对byte数组的操作，使用的是HeapByteBufUtil工具类

protected UnpooledHeapByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
    this(alloc, new byte[initialCapacity], 0, 0, maxCapacity);
}
@Override
public ByteBuf setByte(int index, int value) {
    ensureAccessible();
    _setByte(index, value);
    return this;
}
@Override
protected void _setByte(int index, int value) {
    HeapByteBufUtil.setByte(array, index, value);
}

UnpooledUnsafeHeapByteBuf

该类是非unsafe的子类，继承了大部分的操作，只是操作内存的时候使用的工具类不同，该实现使用的是Unsafe工具类：UnsafeByteBufUtil

UnpooledUnsafeHeapByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
    super(alloc, initialCapacity, maxCapacity);
}
@Override
public ByteBuf setByte(int index, int value) {
    checkIndex(index);
    _setByte(index, value);
    return this;
}

@Override
protected void _setByte(int index, int value) {
    UnsafeByteBufUtil.setByte(array, index, value);
}

堆外内存

UnpooledDirectByteBuf

与堆内内存的不同之处在于此处分配内存使用的是jdk nio的ByteBuffer。直接通过ByteBuffer操作内存

protected UnpooledDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
    super(maxCapacity);
    ...
    this.alloc = alloc;
    setByteBuffer(ByteBuffer.allocateDirect(initialCapacity));
}
@Override
public ByteBuf setByte(int index, int value) {
    ensureAccessible();
    _setByte(index, value);
    return this;
}

@Override
protected void _setByte(int index, int value) {
    buffer.put(index, (byte) value);
}

UnpooledUnsafeDirectByteBuf

堆外内存申请方式与非unsafe方式相同，直接调用jdk nio的ByteBuffer。内存的操作不同是使用UnsafeByteBufUtil工具类

protected UnpooledUnsafeDirectByteBuf(ByteBufAllocator alloc, int initialCapacity, int maxCapacity) {
    super(maxCapacity);
    ...
    this.alloc = alloc;
    setByteBuffer(allocateDirect(initialCapacity));
}
protected ByteBuffer allocateDirect(int initialCapacity) {
    return ByteBuffer.allocateDirect(initialCapacity);
}
@Override
public ByteBuf setBytes(int index, ByteBuf src, int srcIndex, int length) {
    UnsafeByteBufUtil.setBytes(this, addr(index), index, src, srcIndex, length);
    return this;
}
@Override
protected void _setByte(int index, int value) {
    UnsafeByteBufUtil.setByte(addr(index), value);
}

工具类

HeapByteBufUtil

堆内内存操作工具类比较直接，直接对byte数组赋值即可

static void setByte(byte[] memory, int index, int value) {
    memory[index] = (byte) value;
}

UnsafeByteBufUtil

写内存操作通过PlatformDependent类调用，PlatformDependent委派PlatformDependent0调用，PlatformDependent0的静态代码块中获取UNSAFE单例，通过UNSAFE操作堆外内存

static void setByte(long address, int value) {
    PlatformDependent.putByte(address, (byte) value);
}
public static void putByte(long address, byte value) {
    PlatformDependent0.putByte(address, value);
}
static void putByte(long address, byte value) {
    UNSAFE.putByte(address, value);
}
public native void putByte(long var1, byte var3);

池化堆内内存

PooledHeapByteBuf

构造器中根据Recycler与最大容量创建当前实例，并将readerIndex、writerIndex、markedReaderIndex、markedWriterIndex归零

private static final Recycler<PooledHeapByteBuf> RECYCLER = new Recycler<PooledHeapByteBuf>() {
    @Override
    protected PooledHeapByteBuf newObject(Handle handle) {
        return new PooledHeapByteBuf(handle, 0);
    }
};
static PooledHeapByteBuf newInstance(int maxCapacity) {
    PooledHeapByteBuf buf = RECYCLER.get();
    buf.reuse(maxCapacity);
    return buf;
}
PooledHeapByteBuf(Recycler.Handle recyclerHandle, int maxCapacity) {
    super(recyclerHandle, maxCapacity);
}
@Override
protected void _setByte(int index, int value) {
    HeapByteBufUtil.setByte(memory, idx(index), value);
}

向内存写数据，写入的是实例变量memory，查看memory的初始化过程，memory是在父类中初始化，初始化有三处调用分别在PoolChunk、PooledUnsafeDirectByteBuf。暂且关注PoolChunk中的init调用。

void init(PoolChunk<T> chunk, long handle, int offset, int length, int maxLength, PoolThreadCache cache) {
    ...
    memory = chunk.memory;
    ...
}

PoolChunk中初始化调用分别在PoolArena、PoolChunkList、PoolThreadCache。暂且关注PoolArena中的调用。尝试在q050等PoolChunkList中分配，如果失败则添加，创建newChunk内存并初始化，添加至qInit的PoolChunkList，newChunk为抽象方法，PoolArena有俩个子实现类：HeapArena、DirectArena。由于我们此时看的是PooledHeapByteBuf，故暂且关注HeapArena中的实现

private synchronized void allocateNormal(PooledByteBuf<T> buf, int reqCapacity, int normCapacity) {
        ++allocationsNormal;
    if (q050.allocate(buf, reqCapacity, normCapacity) || q025.allocate(buf, reqCapacity, normCapacity) ||
        q000.allocate(buf, reqCapacity, normCapacity) || qInit.allocate(buf, reqCapacity, normCapacity) ||
        q075.allocate(buf, reqCapacity, normCapacity) || q100.allocate(buf, reqCapacity, normCapacity)) {
        return;
    }

    // Add a new chunk.
    PoolChunk<T> c = newChunk(pageSize, maxOrder, pageShifts, chunkSize);
    long handle = c.allocate(normCapacity);
    assert handle > 0;
    c.initBuf(buf, handle, reqCapacity);
    qInit.add(c);
}

不论是池化还是非池化的Chunk创建都是byte数组的封装，非池化的封装仅需要容量。池化的Chunk则需要更多的属性：pageSize、maxOrder、pageShifts、chunkSize

@Override
protected PoolChunk<byte[]> newChunk(int pageSize, int maxOrder, int pageShifts, int chunkSize) {
    return new PoolChunk<byte[]>(this, new byte[chunkSize], pageSize, maxOrder, pageShifts, chunkSize);
}
@Override
protected PoolChunk<byte[]> newUnpooledChunk(int capacity) {
    return new PoolChunk<byte[]>(this, new byte[capacity], capacity);
}

池化堆外内存

PooledUnsafeHeapByteBuf

其父类是PooledHeapByteBuf，区别主要使用了UNSAFE操作内存，其他均一致

@Override
protected void _setByte(int index, int value) {
    UnsafeByteBufUtil.setByte(memory, idx(index), value);
}

PooledDirectByteBuf

初始化过程类似于PooledHeapByteBuf，区别在于PoolArena中创建PoolChunk的newChunk抽象方法的实现类不同，堆外内存使用的DirectArena实现。内存也是使用jdk nio中的ByteBuffer api进行创建。包装的PoolChunk参数也与堆内内存的创建一致

@Override
protected PoolChunk<ByteBuffer> newChunk(int pageSize, int maxOrder, int pageShifts, int chunkSize) {
    return new PoolChunk<ByteBuffer>(
            this, ByteBuffer.allocateDirect(chunkSize), pageSize, maxOrder, pageShifts, chunkSize);
}
@Override
protected PoolChunk<ByteBuffer> newUnpooledChunk(int capacity) {
    return new PoolChunk<ByteBuffer>(this, ByteBuffer.allocateDirect(capacity), capacity);
}

PooledUnsafeDirectByteBuf

类似PooledDirectByteBuf类似，区别在于内存操作使用UNSAFE工具类

Q&A

Unsafe与非Unsafe实现类的应用场景？

池化与非池化的区别？

avatar
文章
阅读
粉丝