【Netty】netty解决TCP粘包/拆包
一、TCP粘包/拆包
1、什么是TCP粘包/拆包
tcp将用户从客户端发往服务端的请求数据包。进行拆分或重新组合进行发送。
例子：
- 将数据包A,拆分成A1,A2两个数据包进行发送。（A1+A2=A,其中A1,A2就是拆包）
- 将数据包A和B进行拆分，然后重组发送。最终发送结果为：A1 , A2+B 其中（A1+A2=A ,B=B，其中A2+B 就是粘包）
2、TCP粘包/拆包发生的原因
- 应用程序write写入的字节大小大于套接口发送缓冲区的大小。
- 进行MSS大小的TCP分段
- 以太网帧的payload大于MTU进行IP分片
3、TCP粘包/拆包发生的原因
由于TCP无法理解上层的业务数据，所以在底层是无法保证数据包不被拆分和重组的。这个问题只能通过上层的应用协议栈设计来解决。根据业界的主流协议的解决方案，可以归纳如下。
- 消息定长，例如每个报文的大小固定长度200字节，如果不够，空位补空格。
- 在包尾增加回车换行符进行分割，例如FTP协议
- 将消息分为消息头和消息体，消息头中包含消息的总长度（或者消息体长度）的字段，通常设计思路为消息头的第一个字段使用int32来表示消息的总长度
- 更复杂的应用层协议
二、时间服务器TCP粘包/拆包的案例
1、按之前写的timerServer代码进行改动。客户端发送100条请求, 服务端接收请求，并响应。
2、服务端代码
package com.spring.test.service.netty.nettydemo.server; import io.netty.bootstrap.ServerBootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelOption; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.nio.NioServerSocketChannel; /** * @date 4:13 PM 2019/8/11 */ public class TimerServer { public static void main(String[] args) { TimerServer timerServer=new TimerServer(); timerServer.bind(8080); } public void bind(int port){ //step1:配置服务端的NIO线程组(一个线程组用于服务端接收客户端连接,一个线程组用于进行socketChannel的网络读写) EventLoopGroup bossGroup=new NioEventLoopGroup(); EventLoopGroup workerGroup=new NioEventLoopGroup(); try { //netty用于启动NIO服务的辅助启动类，目的四降低服务端开发复杂度 ServerBootstrap serverBootstrap=new ServerBootstrap(); serverBootstrap.group(bossGroup,workerGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG,1024) .childHandler(new ChildChannelHandler()); //step2:绑定端口，同步等待成功 ChannelFuture f=serverBootstrap.bind(port).sync(); //step3:等待服务端监听端口关闭 f.channel().closeFuture().sync(); }catch (Exception e){ e.printStackTrace(); }finally { //step4:优雅退出，释放线程池资源 bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } } package com.spring.test.service.netty.nettydemo.server; import io.netty.channel.ChannelInitializer; import io.netty.channel.socket.SocketChannel; /** * @author * @date 4:21 PM 2019/8/11 */ public class ChildChannelHandler extends ChannelInitializer<SocketChannel>{ @Override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new TimerServerHandler()); } } package com.spring.test.service.netty.nettydemo.server; import org.apache.commons.lang.time.DateFormatUtils; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerAdapter; import io.netty.channel.ChannelHandlerContext; /** * @author * @date 4:26 PM 2019/8/11 */ public class TimerServerHandler extends ChannelHandlerAdapter{ private int counter; /** * (接收到第一个数据包的内容如下:) * The time server receive order:QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUE ;the counter is:1 * The time server resp order:BAD ORDER * * * (接收到的第二个数据包内容如下:) * The time server receive order:Y TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER * QUERY TIME ORDER ;the counter is:2 * * The time server resp order:BAD ORDER * @param ctx * @param msg * @throws Exception */ @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { //读取请求 ByteBuf buffer= (ByteBuf) msg; byte[] req=new byte[buffer.readableBytes()]; buffer.readBytes(req); String reqbody=new String(req,"utf-8").substring(0,req.length-System.getProperty("line.separator").length()); System.out.println("The time server receive order:"+reqbody+" ;the counter is:"+ (++counter)); String currentTime="QUERY TIME ORDER".equals(reqbody)? DateFormatUtils.format(System.currentTimeMillis(), "yyyy-MM-dd HH:mm:ss"):"BAD ORDER"; //请求响应 ByteBuf resp=Unpooled.copiedBuffer(currentTime.getBytes()); System.out.println("The time server resp order:"+currentTime); //将相应结果，异步发送给客户端 ctx.write(resp); } @Override public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { //将消息发送队列中的消息写入socketChannel中发送给对方。 ctx.flush(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } }
3、客户端代码
package com.spring.test.service.netty.nettydemo.client; import io.netty.bootstrap.Bootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.ChannelOption; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioSocketChannel; /** * @author * @date 5:41 PM 2019/8/11 */ public class TimerClient { public static void main(String[] args) throws InterruptedException { TimerClient timerClient=new TimerClient(); timerClient.connect(8080,"127.0.0.1"); } public void connect(int port,String host) throws InterruptedException { //step1:配置NIO客户端线程组 EventLoopGroup group=new NioEventLoopGroup(); try { Bootstrap bootstrap=new Bootstrap(); bootstrap.group(group).channel(NioSocketChannel.class) .option(ChannelOption.TCP_NODELAY,true) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { ch.pipeline().addLast(new TimerClientHandler()); } }); //发起异步连接操作(调用同步方法，等待连接成功) ChannelFuture f=bootstrap.connect(host,port).sync(); //等待客户端链路关闭 f.channel().closeFuture().sync(); }catch (Exception e){ e.printStackTrace(); }finally { //优雅的退出，释放NIO线程组 group.shutdownGracefully(); } } } package com.spring.test.service.netty.nettydemo.client; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerAdapter; import io.netty.channel.ChannelHandlerContext; /** * @author * @date 5:49 PM 2019/8/11 */ public class TimerClientHandler extends ChannelHandlerAdapter { private final byte[] firstMessage; private int counter; public TimerClientHandler(){ firstMessage=("QUERY TIME ORDER"+System.getProperty("line.separator")).getBytes(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { System.out.println("释放资源"); ctx.close(); } @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { ByteBuf message=null; //发送请求 for(int i=0;i<100;i++){ message=Unpooled.buffer(firstMessage.length); message.writeBytes(firstMessage); ctx.writeAndFlush(message); } } /** *（接收到服务端的响应如下:） * * Now time is:BAD ORDERBAD ORDER ;the counter is:1 * @param ctx * @param msg * @throws Exception */ @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { //读取响应 ByteBuf buf= (ByteBuf) msg; byte[] resp=new byte[buf.readableBytes()]; buf.readBytes(resp); String responseBody=new String(resp,"utf-8"); System.out.println("Now time is:"+responseBody +" ;the counter is:"+(++counter)); } }
4、现象描述：
- 客户单发送100条消息，服务端只收到2条消息。且发生了对一条消息拆分成2部分的情况。说明客户端在发送时，发生了粘包和拆包的问题。
- 服务端接收到客户端2条消息，响应了两条消息，但客户单只收到了一条响应。说明在服务端向客户端发送响应的时候，发生了粘包的问题。
三、解决时间服务器TCP粘包/拆包的问题
1、在服务端，客户端新增编解码器，解决tcp粘包/拆包的问题
2、服务端代码
package com.spring.test.service.netty.nettydemo.server; import io.netty.bootstrap.ServerBootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelOption; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.nio.NioServerSocketChannel; /** * @date 4:13 PM 2019/8/11 */ public class TimerServer { public static void main(String[] args) { TimerServer timerServer=new TimerServer(); timerServer.bind(8080); } public void bind(int port){ //step1:配置服务端的NIO线程组(一个线程组用于服务端接收客户端连接,一个线程组用于进行socketChannel的网络读写) EventLoopGroup bossGroup=new NioEventLoopGroup(); EventLoopGroup workerGroup=new NioEventLoopGroup(); try { //netty用于启动NIO服务的辅助启动类，目的四降低服务端开发复杂度 ServerBootstrap serverBootstrap=new ServerBootstrap(); serverBootstrap.group(bossGroup,workerGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG,1024) .childHandler(new ChildChannelHandler()); //step2:绑定端口，同步等待成功 ChannelFuture f=serverBootstrap.bind(port).sync(); //step3:等待服务端监听端口关闭 f.channel().closeFuture().sync(); }catch (Exception e){ e.printStackTrace(); }finally { //step4:优雅退出，释放线程池资源 bossGroup.shutdownGracefully(); workerGroup.shutdownGracefully(); } } } package com.spring.test.service.netty.nettydemo.server; import io.netty.channel.ChannelInitializer; import io.netty.channel.socket.SocketChannel; import io.netty.handler.codec.LineBasedFrameDecoder; import io.netty.handler.codec.string.StringDecoder; /** * @author * @date 4:21 PM 2019/8/11 */ public class ChildChannelHandler extends ChannelInitializer<SocketChannel>{ @Override protected void initChannel(SocketChannel socketChannel) throws Exception { //新增编解码器 socketChannel.pipeline().addLast(new LineBasedFrameDecoder(1024)); socketChannel.pipeline().addLast(new StringDecoder()); socketChannel.pipeline().addLast(new TimerServerHandler()); } } package com.spring.test.service.netty.nettydemo.server; import org.apache.commons.lang.time.DateFormatUtils; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerAdapter; import io.netty.channel.ChannelHandlerContext; /** * @author * @date 4:26 PM 2019/8/11 */ public class TimerServerHandler extends ChannelHandlerAdapter{ private int counter; /** * @param ctx * @param msg * @throws Exception */ @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { //读取请求 String body= (String) msg; System.out.println("The time server receive order:"+body+" ;the counter is:"+ (++counter)); String currentTime="QUERY TIME ORDER".equals(body)? DateFormatUtils.format(System.currentTimeMillis(), "yyyy-MM-dd HH:mm:ss"):"BAD ORDER"; currentTime=currentTime+System.getProperty("line.separator"); //请求响应 ByteBuf resp=Unpooled.copiedBuffer(currentTime.getBytes()); System.out.println("The time server resp order:"+currentTime); //将相应结果，异步发送给客户端 ctx.write(resp); } @Override public void channelReadComplete(ChannelHandlerContext ctx) throws Exception { //将消息发送队列中的消息写入socketChannel中发送给对方。 ctx.flush(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { ctx.close(); } }
3、客户端代码
package com.spring.test.service.netty.nettydemo.client; import io.netty.bootstrap.Bootstrap; import io.netty.channel.ChannelFuture; import io.netty.channel.ChannelInitializer; import io.netty.channel.ChannelOption; import io.netty.channel.EventLoopGroup; import io.netty.channel.nio.NioEventLoopGroup; import io.netty.channel.socket.SocketChannel; import io.netty.channel.socket.nio.NioSocketChannel; import io.netty.handler.codec.LineBasedFrameDecoder; import io.netty.handler.codec.string.StringDecoder; /** * @author * @date 5:41 PM 2019/8/11 */ public class TimerClient { public static void main(String[] args) throws InterruptedException { TimerClient timerClient=new TimerClient(); timerClient.connect(8080,"127.0.0.1"); } public void connect(int port,String host) throws InterruptedException { //step1:配置NIO客户端线程组 EventLoopGroup group=new NioEventLoopGroup(); try { Bootstrap bootstrap=new Bootstrap(); bootstrap.group(group).channel(NioSocketChannel.class) .option(ChannelOption.TCP_NODELAY,true) .handler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel ch) throws Exception { //新增编解码器 ch.pipeline().addLast(new LineBasedFrameDecoder(1024)); ch.pipeline().addLast(new StringDecoder()); ch.pipeline().addLast(new TimerClientHandler()); } }); //发起异步连接操作(调用同步方法，等待连接成功) ChannelFuture f=bootstrap.connect(host,port).sync(); //等待客户端链路关闭 f.channel().closeFuture().sync(); }catch (Exception e){ e.printStackTrace(); }finally { //优雅的退出，释放NIO线程组 group.shutdownGracefully(); } } } package com.spring.test.service.netty.nettydemo.client; import io.netty.buffer.ByteBuf; import io.netty.buffer.Unpooled; import io.netty.channel.ChannelHandlerAdapter; import io.netty.channel.ChannelHandlerContext; /** * @author * @date 5:49 PM 2019/8/11 */ public class TimerClientHandler extends ChannelHandlerAdapter { private final byte[] firstMessage; private int counter; public TimerClientHandler() { firstMessage = ("QUERY TIME ORDER" + System.getProperty("line.separator")).getBytes(); } @Override public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception { System.out.println("释放资源"); ctx.close(); } @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { ByteBuf message = null; //发送请求 for (int i = 0; i < 100; i++) { message = Unpooled.buffer(firstMessage.length); message.writeBytes(firstMessage); ctx.writeAndFlush(message); } } /** * * @param ctx * @param msg * @throws Exception */ @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { //读取响应 String resp = (String) msg; System.out.println("Now time is:" + resp + " ;the counter is:" + (++counter)); } }
4、LineBasedFrameDecoder+StringDecoder 如何解决TCP粘包/拆包问题
- LineBasedFrameDecoder：工作原理是它依次遍历ByteBuf中的可读字节，判断看是否有“\n”或者“\r\n”，如果有，就以此位置作为结束位置，从可读索引到结束位置的区间的字节就组成了一行。作为客户端的一条请求消息。它是以换行符为结束标志的解码器。支持携带结束符或者不携带结束符两种解码方式，同时支持配置单行的最大长度。如果连续读取到最大长度后仍然没有发现换行符，就会抛出异常，同时忽略掉之前读到的异常码流。
- StringDecoder：功能非常姜丹，就是将接受到对象转换成字符串，然后调用后面的handler。
- LineBasedFrameDecoder+StringDecoder：组合就是按行切换文本的解码器，它被设计用来支持（上述场景）TCP的粘包和拆包。
四、疑问点
如何发送的消息不是以换行符结束的，该怎么办？或者没有回车换行符，靠消息头中的长度字段来分包怎么办？是不是需要自己写半包解码器？
答案是否定的。Netty提供了多种支持TCP粘包/拆包的解码器，用来满足用户的不同诉求。
五、应用层如何解决TCP粘包/拆包导致的读半包问题
1、消息长度固定，累计读取到长度总和为定长LEN的报文后，就认为读取到一个完整的消息；将计数器置位，重新开始读取下一个数据报。
2、将回车换行符作为消息的结束符，例如FTP协议，这种方式在文本协议中应用比较广泛；
3、将特殊的分隔符作为消息的结束标志，回车换行符就是一种特殊的结束分隔符；
4、通过在消息头中定义长度字段来标识消息的总长度
浙公网安备 33010602011771号