背景知识
如果业务线程调用writeAndFlush()发送消息,会生成WriteAndFlushTask,交由IO线程处理,write操作将消息写入ChannelOutboundBuffer,flush操作将ChannelOutboundBuffer写入socket的发送缓冲区;
ChannelOutboundBuffer是单向链表,没有容量限制;
ChannelOutboundBuffer虽然无界,但是可以给它配置一个高水位线和低水位线,当buffer的大小超过高水位线的时候对应channel的isWritable就会变成false,当buffer的大小低于低水位线的时候,isWritable就会变成true。应用发送数据前应该对isWritable进行判断,防止OOM。高水位线和低水位线是字节数,默认高水位是64K,低水位是32K,通过以下方式进行设置:
.option(ChannelOption.WRITE_BUFFER_HIGH_WATER_MARK, 64 * 1024)
.option(ChannelOption.WRITE_BUFFER_LOW_WATER_MARK, 32 * 1024)
1
2
速率不匹配造成的问题
  如下图所示,当业务线程产生消息的速度大于Socket的发送速率时,首先TCP发送缓冲区会被填满,然后后续所有待发送的数据会不断的占用内存加入到ChannelOutboundBuffer中,出现OOM;


速率不匹配解决方案
  利用ChannelOutboundBuffer的高低水位特性形成闭环链路:当ChannelOutboundBuffer的容量超过高水位时降低消息产生的速度,当ChannelOutboundBuffer的容量小于低水位时增加消息产生的速度,如下图所示:

ChannelOutboundBuffer的容量过高或过低时都会触发fireChannelWritabilityChanged()方法,因此可通过重写channelWritabilityChanged()方法调整消息产生速度,如下所示:

public void channelWritabilityChanged(ChannelHandlerContext ctx) throws Exception {
if(ctx.channel().isWritable()){
//小于低水位,增加速度
}else{
//超过高水位,降低速度
}
}
1
2
3
4
5
6
7
其它策略
速率不匹配时还有其它很多策略,如同步阻塞当前业务线程、丢弃当前消息,或者统计Socket的实际发送速率来调整消息产生速率等,代码示例如下:

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelFutureListener;
import io.netty.channel.ChannelHandlerContext;

public class ChannelWriteUtils {
/**
* 类描述:Socket实际发送Qps统计类
**/
public static class ChannelWriteMetrics {
long startTime;
int qps;
int cnt;

ChannelWriteMetrics() {
startTime = System.currentTimeMillis();
cnt = 0;
}

void inCreaseCnt() {
cnt++;
}

void updateMetrics() {
qps = cnt;
cnt = 1;
startTime = System.currentTimeMillis();
}

public long getStartTime() {
return startTime;
}

public int getQps() {
return qps;
}

public int getCnt() {
return cnt;
}

}

private static Map<ChannelHandlerContext, ChannelWriteMetrics> map = new ConcurrentHashMap<>();

/**
* 策略1:当ChannelOutboundBuffer不可写时产生消息,自适应调整发送速度
**/
public static void processWithMetrics(ChannelHandlerContext ctx, Object msg) {
if (ctx.channel().isWritable()) {
ChannelFuture future = ctx.writeAndFlush(msg);
future.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
ChannelWriteMetrics metrics = map.putIfAbsent(ctx, new ChannelWriteMetrics());
if (System.currentTimeMillis() - metrics.getStartTime() < 1000) {
metrics.inCreaseCnt();
} else {
metrics.updateMetrics();
}
}
});
} else {
ChannelWriteMetrics metrics = map.get(ctx);
int qps = (null != metrics ? metrics.getQps() : 0);
//发送消息(包含Socket实际发送的Qps)
}

}

/**
* 策略2:当ChannelOutboundBuffer不可写时同步阻塞
**/
public static void processWithSync(ChannelHandlerContext ctx, Object msg) {
if (ctx.channel().isWritable()) {
ChannelFuture future = ctx.writeAndFlush(msg);
future.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
if (!future.isSuccess()) {
// 发送失败
}
}
});
} else {
try {
ctx.writeAndFlush(msg).sync();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

/**
* 策略3:当ChannelOutboundBuffer不可写时直接丢弃消息
**/
public static void processWithAbort(ChannelHandlerContext ctx, Object msg) {
if (ctx.channel().isWritable()) {
ctx.writeAndFlush(msg);
}
}
}
————————————————
版权声明:本文为CSDN博主「库昊天」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。
原文链接:https://blog.csdn.net/yangguosb/article/details/79121599

posted on 2021-03-14 00:29  张释文  阅读(512)  评论(1编辑  收藏  举报