七、Netty编解码
- 当
Netty发送或者接受一个消息的时候,就将会发生一次数据转换。入栈消息会被解码:从字节转换为另一种格式(比如java对象);如果是出栈消息,它会被编码成字节。 Netty提供一系列实用的编解码器,他们都实现了ChannelInboundHadnler或者ChannelOutboundHandler接口。在这些类中,channelRead方法已经被重写了。以入栈为例,对于每个从入栈Channel读取的消息,这个方法会被调用。随后,它将调用由解码器所提供的decode()方法进行解码,并将已经解码的字节转发给ChannelPipeline中的下一个ChannelInboundHandler。
7.1 常用编解码器
7.1.1 ByteToMessageDecoder、MessageToByteEncoder
-
解码器-
ByteToMessageDecoder常用的解码器实现如下图
-
编码器-
MessageToByteEncoder编码器相对简单,只需要将二进制字节流写入ByteBuf中即可;举例说明如下图
7.2 TCP粘包、拆包问题
7.2.1 基本介绍
-
TCP是面向连接的,面向流的,提供高可靠性服务。收发两端(客户端和服务器端)都要有一一成对的socket,因此,发送端为了将多个发给接收端的包,更有效的发给对方,使用了优化方法(Nagle算法),将多次间隔较小且数据量小的数据,合并成一个大的数据块,然后进行封包。这样做虽然提高了效率,但是接收端就难于分辨出完整的数据包了,因为面向流的通信是无消息保护边界的 -
由于
TCP无消息保护边界,需要在接收端处理消息边界问题,也就是我们所说的粘包、拆包问题,如下图
对上图的说明:假设客户端分别发送了两个数据包 D1 和 D2 给服务端,由于服务端一次读取到字节数是不确定的,故可能存在以下四种情况:
- 服务端分两次读取到了两个独立的数据包,分别是
D1和D2,没有粘包和拆包 - 服务端一次接受到了两个数据包,
D1和D2粘合在一起,称之为TCP粘包 - 服务端分两次读取到了数据包,第一次读取到了完整的
D1包和D2包的部分内容,第二次读取到了D2包的剩余内容,这称之为TCP拆包 - 服务端分两次读取到了数据包,第一次读取到了
D1包的部分内容D1_1,第二次读取到了D1包的剩余部分内容D1_2和完整的D2包。
7.2.2 编码实例(演示粘包、拆包)
现象展示:
-
启动服务端,分别启动三个客户端
-
观察到服务端收到三个客户端发送的消息格式各不相同,但事实上三个客户端发送的消息都是一样的,这边就出现了粘包的现象
三个客户端收到的消息格式也各不相同
代码如下:
-
服务端
package com.nic.netty.pack; import io.netty.bootstrap.ServerBootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.ChannelOption; import io.netty.channel.ChannelPipeline; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioServerSocketChannel; /** * Description: * 拆包粘包演示服务端 * * @author james * @date 2021/7/23 10:01 */ public class PackServer { private int port; public PackServer(int port) { this.port = port; } public static void main(String[] args) { new PackServer(7200).listen(); } public void listen() { NioEventLoopGroup bossGroup = new NioEventLoopGroup(1); NioEventLoopGroup workerGroup = new NioEventLoopGroup(4); try { ServerBootstrap serverBootstrap = new ServerBootstrap(); serverBootstrap.group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG, 128) .childOption(ChannelOption.SO_KEEPALIVE, true) .childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ChannelPipeline pipeline = ch.pipeline(); pipeline .addLast(new PackServerHandler()); } }); System.out.println("netty 服务器 is ready..."); ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); channelFuture.channel().closeFuture().sync(); } catch (Exception e) { e.printStackTrace(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } } -
服务端自定义handler
package com.nic.netty.pack; import cn.hutool.core.util.RandomUtil; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; /** * Description: * 拆包粘包演示服务端自定义handler * * @author james * @date 2021/7/23 10:02 */ public class PackServerHandler extends SimpleChannelInboundHandler<ByteBuf> { private int count; @Override protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception { byte[] bytes = new byte[msg.readableBytes()]; msg.readBytes(bytes); String message = new String(bytes, CharsetUtil.UTF_8); System.out.println("服务端收到消息:" + message); System.out.println("服务端收到消息数量:" + (++this.count)); //服务端收到消息之后,回写客户端 ByteBuf respBuf = Unpooled.copiedBuffer(RandomUtil.randomString(10) + " ", CharsetUtil.UTF_8); ctx.writeAndFlush(respBuf); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } } -
客户端
package com.nic.netty.pack; import io.netty.bootstrap.Bootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.ChannelPipeline; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioSocketChannel; /** * Description: * 拆包粘包演示客户端 * * @author james * @date 2021/7/23 9:58 */ public class PackClient { private String host; private int port; public PackClient(String host, int port) { this.host = host; this.port = port; } public static void main(String[] args) { new PackClient("127.0.0.1", 7200).run(); } public void run() { NioEventLoopGroup group = new NioEventLoopGroup(); try { Bootstrap bootstrap = new Bootstrap(); bootstrap.group(group) .channel(NioSocketChannel.class) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ChannelPipeline pipeline = ch.pipeline(); pipeline.addLast(new PackClientHandler()); } }); ChannelFuture channelFuture = bootstrap.connect(host, port).sync(); channelFuture.channel().closeFuture().sync(); } catch (Exception e) { e.printStackTrace(); } finally { group.shutdownGracefully(); } } } -
客户端自定义handler
package com.nic.netty.pack; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; /** * Description: * 拆包粘包演示客户端自定义handler * * @author james * @date 2021/7/23 9:55 */ public class PackClientHandler extends SimpleChannelInboundHandler<ByteBuf> { private int count; @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { System.out.println("channelActive"); for (int i = 0; i < 10; i++) { ByteBuf byteBuf = Unpooled.copiedBuffer("hello,server " + i + " ", CharsetUtil.UTF_8); ctx.writeAndFlush(byteBuf); } } @Override protected void channelRead0(ChannelHandlerContext ctx, ByteBuf msg) throws Exception { byte[] bytes = new byte[msg.readableBytes()]; msg.readBytes(bytes); String message = new String(bytes, CharsetUtil.UTF_8); System.out.println("客户端收到消息:" + message); System.out.println("客户端收到消息数量:" + (++this.count)); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } }
7.3 Netty编解码解决粘包、拆包问题
解决方案:
- 使用自定义协议+编解码器来解决
- 关键就是要解决服务器端每次读取数据长度的问题,这个问题解决,就不会出现服务器多读或少读数据的问题,从而避免的
TCP粘包、拆包。
启动服务端,分别启动三个客户端,看到服务端收到的消息是正常的了,没有出现粘包拆包现象
客户端收到服务端回写的消息也是正常的
代码如下:
-
服务端和客户端只需要修改pipeline中的自定义handler即可
-
消息协议
package com.nic.netty.pack; /** * Description: * 自定义消息协议,解决粘包拆包问题 * * @author james * @date 2021/7/23 10:28 */ public class MessageProtocol { //数据包长度 private int len; private byte[] content; public MessageProtocol(int len, byte[] content) { this.len = len; this.content = content; } public int getLen() { return len; } public void setLen(int len) { this.len = len; } public byte[] getContent() { return content; } public void setContent(byte[] content) { this.content = content; } } -
消息编码
package com.nic.netty.pack; import io.netty.buffer.ByteBuf; import io.netty.channel.ChannelHandlerContext; import io.netty.handler.codec.MessageToByteEncoder; /** * Description: * * @author james * @date 2021/7/23 10:29 */ public class MessageEncoder extends MessageToByteEncoder<MessageProtocol> { @Override protected void encode(ChannelHandlerContext ctx, MessageProtocol msg, ByteBuf out) throws Exception { System.out.println("MessageEncoder..."); out.writeInt(msg.getLen()); out.writeBytes(msg.getContent()); } } -
消息解码
package com.nic.netty.pack; import io.netty.buffer.ByteBuf; import io.netty.channel.ChannelHandlerContext; import io.netty.handler.codec.ReplayingDecoder; import java.util.List; /** * Description: * ReplayingDecoder 扩展了 ByteToMessageDecoder 类, * 使用这个类,我们不必调用 readableBytes() 方法。参数 S 指定了用户状态管理的类型,其中 Void 代表不需要状态管理 * * @author james * @date 2021/7/23 10:32 */ public class MessageDecoder extends ReplayingDecoder<Void> { @Override protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception { System.out.println("MessageDecoder..."); //获取MessageProtocol数据包的长度 int length = in.readInt(); byte[] bytes = new byte[length]; in.readBytes(bytes); //封装MessageProtocol对象,放入out,传递给下一个handler MessageProtocol messageProtocol = new MessageProtocol(length, bytes); out.add(messageProtocol); } } -
服务端自定义handler
package com.nic.netty.pack; import cn.hutool.core.util.RandomUtil; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; import java.nio.charset.StandardCharsets; /** * Description: * 拆包粘包演示服务端自定义handler,自定义消息协议 * * @author james * @date 2021/7/23 10:02 */ public class PackServerHandler2 extends SimpleChannelInboundHandler<MessageProtocol> { private int count; @Override protected void channelRead0(ChannelHandlerContext ctx, MessageProtocol msg) throws Exception { int len = msg.getLen(); byte[] content = msg.getContent(); String message = new String(content, CharsetUtil.UTF_8); System.out.println("服务端收到消息长度:" + len + ",消息内容:" + message + ",消息数量:" + (++this.count)); //服务端收到消息之后,回写客户端 //会写客户端需要转成MessageProtocol String randomString = RandomUtil.randomString(10); byte[] respContent = randomString.getBytes(StandardCharsets.UTF_8); MessageProtocol messageProtocol = new MessageProtocol(respContent.length, respContent); ctx.writeAndFlush(messageProtocol); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } } -
客户端自定义handler
package com.nic.netty.pack; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.SimpleChannelInboundHandler; import io.netty.util.CharsetUtil; import java.nio.charset.StandardCharsets; /** * Description: * 拆包粘包演示客户端自定义handler,自定义消息协议 * * @author james * @date 2021/7/23 10:37 */ public class PackClientHandler2 extends SimpleChannelInboundHandler<MessageProtocol> { private int count; @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { System.out.println("channelActive"); for (int i = 0; i < 10; i++) { //这边写数据需要转成MessageProtocol String msg = "hello,server " + i; byte[] content = msg.getBytes(StandardCharsets.UTF_8); MessageProtocol messageProtocol = new MessageProtocol(content.length, content); ctx.writeAndFlush(messageProtocol); } } @Override protected void channelRead0(ChannelHandlerContext ctx, MessageProtocol msg) throws Exception { int len = msg.getLen(); byte[] content = msg.getContent(); String message = new String(content, CharsetUtil.UTF_8); System.out.println("客户端收到消息长度:" + len + ",消息内容:" + message + ",消息数量:" + (++this.count)); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { System.out.println(cause.getMessage()); ctx.close(); } }
八、Netty实现Dubbo RPC
8.1 RPC基本介绍
RPC(Remote Procedure Call)—远程过程调用,是一个计算机通信协议。该协议允许运行于一台计算机的程序调用另一台计算机的子程序,而程序员无需额外地为这个交互作用编程- 两个或多个应用程序都分布在不同的服务器上,它们之间的调用都像是本地方法调用一样
8.2 RPC调用流程
- 服务消费方(
client)以本地调用方式调用服务 client stub接收到调用后负责将方法、参数等封装成能够进行网络传输的消息体client stub将消息进行编码并发送到服务端server stub收到消息后进行解码server stub根据解码结果调用本地的服务- 本地服务执行并将结果返回给
server stub server stub将返回导入结果进行编码并发送至消费方client stub接收到消息并进行解码- 服务消费方(
client)得到结果
小结:RPC 的目标就是将 2 - 8 这些步骤都封装起来,用户无需关心这些细节,可以像调用本地方法一样即可完成远程服务调用
8.3 编码实例
- 创建一个接口
IHelloService,定义抽象方法。用于消费者和提供者之间的约定。 - 创建一个提供者
RpcNettyServer,该类需要监听消费者的请求,并按照约定返回数据。 - 创建一个消费者
RpcNettyClient,该类需要透明的调用自己不存在的方法,内部需要使用Netty请求提供者返回数据 - 创建服务端调用实例
ServerBootstrap和客户端调用实例ClientBootstrap,通过ClientBootstrap调用公共接口发送消息给服务端。
测试流程:
-
分别启动
ServerBootstrap和ClientBootstrap -
服务端会受到客户端每隔2s发送的消息
-
客户端会受到服务端回复的消息
代码如下:
-
公共接口service
package com.nic.netty.rpc.service; /** * Description: * * @author james * @date 2021/7/23 13:42 */ public interface IHelloService { String hello(String msg); } -
客户端
package com.nic.netty.rpc.handler; import cn.hutool.core.thread.ThreadFactoryBuilder; import io.netty.bootstrap.Bootstrap; import io.netty.channel.ChannelInitializer; import io.netty.channel.ChannelOption; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioSocketChannel; import io.netty.handler.codec.string.StringDecoder; import io.netty.handler.codec.string.StringEncoder; import java.lang.reflect.InvocationHandler; import java.lang.reflect.Method; import java.lang.reflect.Proxy; import java.util.concurrent.ExecutorService; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.ThreadPoolExecutor; import java.util.concurrent.TimeUnit; /** * Description: * * @author james * @date 2021/7/23 14:04 */ public class RpcNettyClient { private static final ExecutorService POOL = new ThreadPoolExecutor(1, 200, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(1024), new ThreadFactoryBuilder().setNamePrefix("nettyClient-pool-%d").build(), new ThreadPoolExecutor.AbortPolicy()); private static RpcNettyClientHandler clientHandler; private int count; public Object getBean(final Class<?> serviceClass, final String providerName) { return Proxy.newProxyInstance(Thread.currentThread().getContextClassLoader(), new Class<?>[] {serviceClass}, new InvocationHandler() { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { System.out.println("(proxy,method,args) 进入..." + (++count) + "次"); if (null == clientHandler) { initClient(); } clientHandler.setParam(providerName + args[0]); return POOL.submit(clientHandler).get(); } }); } public static void initClient() { clientHandler = new RpcNettyClientHandler(); NioEventLoopGroup group = new NioEventLoopGroup(); Bootstrap bootstrap = new Bootstrap(); bootstrap.group(group) .channel(NioSocketChannel.class) .option(ChannelOption.TCP_NODELAY, true) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ch.pipeline() .addLast(new StringDecoder()) .addLast(new StringEncoder()) .addLast(clientHandler); } }); try { bootstrap.connect("127.0.0.1", 7300).sync(); } catch (InterruptedException e) { e.printStackTrace(); } } } -
客户端自定义handler
package com.nic.netty.rpc.handler; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.ChannelInboundHandlerAdapter; import java.util.concurrent.Callable; /** * Description: * * @author james * @date 2021/7/23 13:59 */ public class RpcNettyClientHandler extends ChannelInboundHandlerAdapter implements Callable { //上下文 private ChannelHandlerContext context; //返回结果 private String result; //客户端调用方法时传入的参数 private String param; @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { System.out.println("channelActive..."); context = ctx; } //synchronized //同步处理,读完数据之后notify @Override public synchronized void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { System.out.println("3.channelRead..."); result = msg.toString(); notify(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } @Override public synchronized Object call() throws Exception { System.out.println("2.call 1 ..."); context.writeAndFlush(param); wait(); System.out.println("4.call 2 ..."); return result; } void setParam(String param) { System.out.println("1.setParam..."); this.param = param; } } -
服务端
package com.nic.netty.rpc.handler; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioServerSocketChannel; import io.netty.handler.codec.string.StringDecoder; import io.netty.handler.codec.string.StringEncoder; /** * Description: * * @author james * @date 2021/7/23 14:29 */ public class RpcNettyServer { private int port; public RpcNettyServer(int port) { this.port = port; } public void listen() { NioEventLoopGroup bossGroup = new NioEventLoopGroup(1); NioEventLoopGroup workerGroup = new NioEventLoopGroup(); try { io.netty.bootstrap.ServerBootstrap serverBootstrap = new io.netty.bootstrap.ServerBootstrap(); serverBootstrap.group(bossGroup, workerGroup) .channel(NioServerSocketChannel.class) .childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ch.pipeline() .addLast(new StringDecoder()) .addLast(new StringEncoder()) .addLast(new RpcNettyServerHandler()); } }); ChannelFuture channelFuture = serverBootstrap.bind(port).sync(); System.out.println("rpcNetty 服务器 is ready..."); channelFuture.channel().closeFuture().sync(); } catch (Exception e) { e.printStackTrace(); } finally { bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } } -
服务端自定义handler
package com.nic.netty.rpc.handler; import com.nic.netty.rpc.consumer.ClientBootstrap; import com.nic.netty.rpc.provider.HelloServiceImpl; import io.netty.channel.ChannelHandlerContext; import io.netty.channel.ChannelInboundHandlerAdapter; /** * Description: * * @author james * @date 2021/7/23 13:54 */ public class RpcNettyServerHandler extends ChannelInboundHandlerAdapter { @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { System.out.println("msg = " + msg); //客户端调用服务器api时,需要定义一个协议 //比如我们要求每次发消息都必须以某个字符串为开头"HelloRPC#hello#" if (msg.toString().startsWith(ClientBootstrap.PROVICER_NAME)) { String result = new HelloServiceImpl() .hello(msg.toString().substring(msg.toString().lastIndexOf("#") + 1)); ctx.writeAndFlush(result); } } } -
接口service实现
package com.nic.netty.rpc.provider; import com.nic.netty.rpc.service.IHelloService; import org.apache.commons.lang3.StringUtils; /** * Description: * * @author james * @date 2021/7/23 14:34 */ public class HelloServiceImpl implements IHelloService { private static int count = 0; @Override public String hello(String msg) { System.out.println("收到客户的消息:" + msg); String respMsg = "你好客户端,我已经收到你的消息"; if (StringUtils.isNotBlank(msg)) { return String.format("%s [%s] 第%s次", respMsg, msg, (++count)); } else { return respMsg; } } } -
服务端提供者
package com.nic.netty.rpc.provider; import com.nic.netty.rpc.handler.RpcNettyServer; /** * Description: * * @author james * @date 2021/7/23 13:50 */ public class ServerBootstrap { public static void main(String[] args) { new RpcNettyServer(7300).listen(); } } -
客户端消费者
package com.nic.netty.rpc.consumer; import cn.hutool.core.date.DateUtil; import com.nic.netty.rpc.handler.RpcNettyClient; import com.nic.netty.rpc.service.IHelloService; /** * Description: * * @author james * @date 2021/7/23 14:17 */ public class ClientBootstrap { public static final String PROVICER_NAME = "helloRPC#hello#"; public static void main(String[] args) throws InterruptedException { RpcNettyClient consumer = new RpcNettyClient(); IHelloService service = (IHelloService) consumer.getBean(IHelloService.class, PROVICER_NAME); while (true) { Thread.sleep(2 * 1000); String result = service.hello(DateUtil.now() + " 你好 dubbo!"); System.out.println("调用结果:" + result); } } }
完整代码
https://github.com/beiJxx/netty_test
资料参考