Sentinel - 规则持久化
1. Sentinel规则推送模式
|
推送模式
|
说明
|
优点
|
缺点
|
|
原始模式
|
API 将规则推送至客户端并直接更新到内存中,扩展写数据源(WritableDataSource)
|
简单,无任何依赖
|
不保证一致性;规则保存在内存中,重启即消失。严重不建议用于生产环境
|
|
Pull 模式
|
扩展写数据源(WritableDataSource), 客户端主动向某个规则管理中心定期轮询拉取规则,这个规则中心可以是 RDBMS、文件 等
|
简单,无任何依赖;规则持久化
|
不保证一致性;实时性不保证,拉取过于频繁也可能会有性能问题。
|
|
Push 模式
|
扩展读数据源(ReadableDataSource),规则中心统一推送,客户端通过注册监听器的方式时刻监听变化,比如使用 Nacos、Zookeeper 等配置中心。这种方式有更好的实时性和一致性保证。生产环境下一般采用 push 模式的数据源。
|
规则持久化;一致性;快速
|
引入第三方依赖
|
1.1 原始模式
rule规则是如何放到内存中的?
loadRules(rules) -> currentProperty.updateValue(rules) -> flowRules.putAll(rules);
结构时Map结构,用来存储rule到内存结构中:Map<String, List<FlowRule>> flowRules = new ConcurrentHashMap<String, List<FlowRule>>();
看controller,看一下哪里有扩展点,哪里可以将rule进行持久化?如何和 DashBoard 进行通信
DispatchServlet#doDispatch
发布流控配置:
/v1/flow/rule,dashboard的一些参数会封装为FlowRuleEntity对象。
控制台保存规则: entry = repository.save(entry);
--- 扩展点,可以在这里往mysql,redis存数据。repository是基于内存的,它继承了RuleRepository
publishRules(entry.getApp(), entry.getIp(), entry.getPort()).get(5000, TimeUnit.MILLSECONDS);
--->
List<FlowRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
sentinelApiClient.setFlowRuleOfMachineAsync(app, ip, port, rules);
FlowRuleEntity -----> FlowRule toRule
发起请求设置流控规则:
http://192.168.xx.xx:8719/setRules
客户端:
CommandCenter.start()
根据CommandHandler 找到 Flow类型的Handler,然后获取写数据源,持久化:
ModifyRulesCommandHandler#writeToDataSource
拉规则如何集成到springCloud中?
1. ApplicationListener
2. @PostConstructor
3. BeanPostProcessor
4. SmartInitializingSingleton
5. SpringBoot:ApplicationRunner
优先考虑Sentinel是否可以实现扩展?
拉模式:
SPI机制:initFunc 在这里可以实现拉模式的读写数据源的配置。
监听变化,更新到内存中
sentinel properties 绑定了 监听器,比如FlowRuleManager里面有一个监听器,FlowPropertyListener 就是将配置更新到内存中的。
推模式:
NacosDatasource , 它在sentinel-datasource-nacos包下。
如果需要引入推模式,需要引入sentinel-datasource-nacos依赖。
还需要引入sentinel-datasource-extension的依赖,因为这里面有读数据源和写数据源的接口。
NacosDatasource继承了AbstractDatasource,它就在extension包下面。
控制台与Client端通信参考:
https://www.processon.com/view/link/639c34fe1e085317e0a143ae
1.2 拉模式
<dependency> <groupId>com.alibaba.csp</groupId> <artifactId>sentinel-datasource-extension</artifactId> <version>1.8.0</version> </dependency>
// FileRefreshableDataSource 会周期性的读取文件以获取规则,当文件有更新时会及时发现,并将规则更新到内存中。 ReadableDataSource<String, List<FlowRule>> ds = new FileRefreshableDataSource<>( flowRulePath, source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {}) ); // 将可读数据源注册至 FlowRuleManager. FlowRuleManager.register2Property(ds.getProperty()); WritableDataSource<List<FlowRule>> wds = new FileWritableDataSource<>(flowRulePath, this::encodeJson); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. WritableDataSourceRegistry.registerFlowDataSource(wds);
其中部分核心代码:
public class FileDataSourceInit implements InitFunc { @Override public void init() throws Exception { //创建文件存储目录 RuleFileUtils.mkdirIfNotExits(PersistenceRuleConstant.storePath); //创建规则文件 RuleFileUtils.createFileIfNotExits(PersistenceRuleConstant.rulesMap); //处理流控规则逻辑 dealFlowRules(); // 处理降级规则 dealDegradeRules(); // 处理系统规则 dealSystemRules(); // 处理热点参数规则 dealParamFlowRules(); // 处理授权规则 dealAuthRules(); } private void dealFlowRules() throws FileNotFoundException { String ruleFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.FLOW_RULE_PATH).toString(); //创建流控规则的可读数据源 ReadableDataSource<String, List<FlowRule>> flowRuleRDS = new FileRefreshableDataSource( ruleFilePath, RuleListConverterUtils.flowRuleListParser ); // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时,就会更新规则到内存 FlowRuleManager.register2Property(flowRuleRDS.getProperty()); WritableDataSource<List<FlowRule>> flowRuleWDS = new FileWritableDataSource<List<FlowRule>>( ruleFilePath, RuleListConverterUtils.flowFuleEnCoding ); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. WritableDataSourceRegistry.registerFlowDataSource(flowRuleWDS); } private void dealDegradeRules() throws FileNotFoundException { //讲解规则文件路径 String degradeRuleFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.DEGRAGE_RULE_PATH).toString(); //创建流控规则的可读数据源 ReadableDataSource<String, List<DegradeRule>> degradeRuleRDS = new FileRefreshableDataSource( degradeRuleFilePath, RuleListConverterUtils.degradeRuleListParse ); // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时,就会更新规则到内存 DegradeRuleManager.register2Property(degradeRuleRDS.getProperty()); WritableDataSource<List<DegradeRule>> degradeRuleWDS = new FileWritableDataSource<>( degradeRuleFilePath, RuleListConverterUtils.degradeRuleEnCoding ); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. WritableDataSourceRegistry.registerDegradeDataSource(degradeRuleWDS); } private void dealSystemRules() throws FileNotFoundException { //讲解规则文件路径 String systemRuleFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.SYSTEM_RULE_PATH).toString(); //创建流控规则的可读数据源 ReadableDataSource<String, List<SystemRule>> systemRuleRDS = new FileRefreshableDataSource( systemRuleFilePath, RuleListConverterUtils.sysRuleListParse ); // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时,就会更新规则到内存 SystemRuleManager.register2Property(systemRuleRDS.getProperty()); WritableDataSource<List<SystemRule>> systemRuleWDS = new FileWritableDataSource<>( systemRuleFilePath, RuleListConverterUtils.sysRuleEnCoding ); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. WritableDataSourceRegistry.registerSystemDataSource(systemRuleWDS); } private void dealParamFlowRules() throws FileNotFoundException { //讲解规则文件路径 String paramFlowRuleFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.HOT_PARAM_RULE).toString(); //创建流控规则的可读数据源 ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleRDS = new FileRefreshableDataSource( paramFlowRuleFilePath, RuleListConverterUtils.paramFlowRuleListParse ); // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时,就会更新规则到内存 ParamFlowRuleManager.register2Property(paramFlowRuleRDS.getProperty()); WritableDataSource<List<ParamFlowRule>> paramFlowRuleWDS = new FileWritableDataSource<>( paramFlowRuleFilePath, RuleListConverterUtils.paramRuleEnCoding ); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. ModifyParamFlowRulesCommandHandler.setWritableDataSource(paramFlowRuleWDS); } private void dealAuthRules() throws FileNotFoundException { //讲解规则文件路径 String authFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.AUTH_RULE_PATH).toString(); //创建流控规则的可读数据源 ReadableDataSource<String, List<AuthorityRule>> authRuleRDS = new FileRefreshableDataSource( authFilePath, RuleListConverterUtils.authorityRuleParse ); // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时,就会更新规则到内存 AuthorityRuleManager.register2Property(authRuleRDS.getProperty()); WritableDataSource<List<AuthorityRule>> authRuleWDS = new FileWritableDataSource<>( authFilePath, RuleListConverterUtils.authorityEncoding ); // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中. // 这样收到控制台推送的规则时,Sentinel 会先更新到内存,然后将规则写入到文件中. WritableDataSourceRegistry.registerAuthorityDataSource(authRuleWDS); } }
改完之后,打个包,build到本地仓库,自己的微服务引入这个依赖,就ok啦。
1.3 推模式
1.3.1 基于Nacos配置中心控制台实现推送
<dependency> <groupId>com.alibaba.csp</groupId> <artifactId>sentinel-datasource-nacos</artifactId> <version>1.8.0</version> </dependency>
// nacos server ip private static final String remoteAddress = "localhost:8848"; // nacos group private static final String groupId = "Sentinel:Demo"; // nacos dataId private static final String dataId = "com.alibaba.csp.sentinel.demo.flow.rule"; ReadableDataSource<String, List<FlowRule>> flowRuleDataSource = new NacosDataSource<>(remoteAddress, groupId, dataId,source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {})); FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
[ { "resource": "TestResource", "controlBehavior": 0, "count": 10.0, "grade": 1, "limitApp": "default", "strategy": 0 } ]

<!--sentinel持久化 采用 Nacos 作为规则配置数据源--> <dependency> <groupId>com.alibaba.csp</groupId> <artifactId>sentinel-datasource-nacos</artifactId> </dependency>
spring:
application:
name: mall-user-sentinel-demo
cloud:
nacos:
discovery:
server-addr: 127.0.0.1:8848
sentinel:
transport:
# 添加sentinel的控制台地址
dashboard: 127.0.0.1:8080
# 指定应用与Sentinel控制台交互的端口,应用本地会起一个该端口占用的HttpServer
port: 8719
datasource:
ds1:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}
groupId: DEFAULT_GROUP
data-type: json
rule-type: flow
3)nacos配置中心中添加
[
{
"resource": "userinfo",
"limitApp": "default",
"grade": 1,
"count": 1,
"strategy": 0,
"controlBehavior": 0,
"clusterMode": false
}
]

缺点:直接在Sentinel Dashboard中修改规则配置,配置中心的配置不会发生变化
1.3.2 基于Sentinel控制台实现推送
- DynamicRuleProvider: 拉取规则
- DynamicRulePublisher: 推送规则

注意:微服务接入Sentinel client,yml配置需要匹配对应的规则后缀

第2步:进入com.alibaba.csp.sentinel.dashboard.controller包下修改对应的规则controller实现类

以流控规则为例,从Nacos配置中心获取所有的流控规则。
@GetMapping("/rules")
@AuthAction(PrivilegeType.READ_RULE)
public Result<List<FlowRuleEntity>> apiQueryMachineRules(@RequestParam String app,
@RequestParam String ip,
@RequestParam Integer port) {
if (StringUtil.isEmpty(app)) {
return Result.ofFail(-1, "app can't be null or empty");
}
if (StringUtil.isEmpty(ip)) {
return Result.ofFail(-1, "ip can't be null or empty");
}
if (port == null) {
return Result.ofFail(-1, "port can't be null");
}
try {
// List<FlowRuleEntity> rules = sentinelApiClient.fetchFlowRuleOfMachine(app, ip, port);
//从配置中心获取规则配置
List<FlowRuleEntity> rules = ruleProvider.getRules(app, ip, port);
rules = repository.saveAll(rules);
return Result.ofSuccess(rules);
} catch (Throwable throwable) {
logger.error("Error when querying flow rules", throwable);
return Result.ofThrowable(-1, throwable);
}
}
@PostMapping("/rule")
@AuthAction(PrivilegeType.WRITE_RULE)
public Result<FlowRuleEntity> apiAddFlowRule(@RequestBody FlowRuleEntity entity) {
Result<FlowRuleEntity> checkResult = checkEntityInternal(entity);
if (checkResult != null) {
return checkResult;
}
entity.setId(null);
Date date = new Date();
entity.setGmtCreate(date);
entity.setGmtModified(date);
entity.setLimitApp(entity.getLimitApp().trim());
entity.setResource(entity.getResource().trim());
try {
entity = repository.save(entity);
//publishRules(entity.getApp(), entity.getIp(), entity.getPort()).get(5000, TimeUnit.MILLISECONDS);
//发布规则到配置中心
publishRules(entity.getApp());
return Result.ofSuccess(entity);
} catch (Throwable t) {
Throwable e = t instanceof ExecutionException ? t.getCause() : t;
logger.error("Failed to add new flow rule, app={}, ip={}", entity.getApp(), entity.getIp(), e);
return Result.ofFail(-1, e.getMessage());
}
}
/**
* 发布到配置中心
* @param app
* @throws Exception
*/
private void publishRules(/*@NonNull*/ String app) throws Exception {
List<FlowRuleEntity> rules = repository.findAllByApp(app);
rulePublisher.publish(app, rules);
}
<!--sentinel持久化 采用 Nacos 作为规则配置数据源--> <dependency> <groupId>com.alibaba.csp</groupId> <artifactId>sentinel-datasource-nacos</artifactId> </dependency>
server:
port: 8806
spring:
application:
name: mall-user-sentinel-rule-push-demo #微服务名称
#配置nacos注册中心地址
cloud:
nacos:
discovery:
server-addr: 127.0.0.1:8848
sentinel:
transport:
# 添加sentinel的控制台地址
dashboard: 127.0.0.1:8080
# 指定应用与Sentinel控制台交互的端口,应用本地会起一个该端口占用的HttpServer
#port: 8719
datasource:
# ds1: #名称自定义,唯一
# nacos:
# server-addr: 127.0.0.1:8848
# dataId: ${spring.application.name}
# groupId: DEFAULT_GROUP
# data-type: json
# rule-type: flow
flow-rules:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}-flow-rules
groupId: SENTINEL_GROUP # 注意groupId对应Sentinel Dashboard中的定义
data-type: json
rule-type: flow
degrade-rules:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}-degrade-rules
groupId: SENTINEL_GROUP
data-type: json
rule-type: degrade
param-flow-rules:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}-param-flow-rules
groupId: SENTINEL_GROUP
data-type: json
rule-type: param-flow
authority-rules:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}-authority-rules
groupId: SENTINEL_GROUP
data-type: json
rule-type: authority
system-rules:
nacos:
server-addr: 127.0.0.1:8848
dataId: ${spring.application.name}-system-rules
groupId: SENTINEL_GROUP
data-type: json
rule-type: system
热点参数规则失效和解决思路
[{
"app": "mall-user-sentinel-rule-push-demo",
"gmtCreate": 1616136838785,
"gmtModified": 1616136838785,
"id": 1,
"ip": "192.168.3.1",
"port": 8719,
"rule": {
"burstCount": 0,
"clusterConfig": {
"fallbackToLocalWhenFail": true,
"sampleCount": 10,
"thresholdType": 0,
"windowIntervalMs": 1000
},
"clusterMode": false,
"controlBehavior": 0,
"count": 1.0,
"durationInSec": 1,
"grade": 1,
"limitApp": "default",
"maxQueueingTimeMs": 0,
"paramFlowItemList": [],
"paramIdx": 1,
"resource": "hot"
}
}, {
"app": "mall-user-sentinel-rule-push-demo",
"gmtCreate": 1616137178470,
"gmtModified": 1616658923519,
"id": 2,
"ip": "192.168.3.1",
"port": 8719,
"rule": {
"burstCount": 0,
"clusterConfig": {
"fallbackToLocalWhenFail": true,
"sampleCount": 10,
"thresholdType": 0,
"windowIntervalMs": 1000
},
"clusterMode": false,
"controlBehavior": 0,
"count": 3.0,
"durationInSec": 1,
"grade": 1,
"limitApp": "default",
"maxQueueingTimeMs": 0,
"paramFlowItemList": [{
"classType": "int",
"count": 1,
"object": "4"
}],
"paramIdx": 0,
"resource": "findOrderByUserId"
}
}]
@Configuration public class ConverterConfig { @Bean("sentinel-json-param-flow-converter2") @Primary public JsonConverter jsonParamFlowConverter() { return new FlowParamJsonConverter(new ObjectMapper(), ParamFlowRule.class); } } @Component public class FlowParamConverterBeanPostProcessor implements BeanPostProcessor { @Autowired private JsonConverter jsonParamFlowConverter; @Override public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException { if (beanName.equals("param-flow-rules-sentinel-nacos-datasource")) { NacosDataSourceFactoryBean nacosDataSourceFactoryBean = (NacosDataSourceFactoryBean) bean; nacosDataSourceFactoryBean.setConverter(jsonParamFlowConverter); return bean; } return bean; } } public class FlowParamJsonConverter extends JsonConverter { Class ruleClass; public FlowParamJsonConverter(ObjectMapper objectMapper, Class ruleClass) { super(objectMapper, ruleClass); this.ruleClass = ruleClass; } @Override public Collection<Object> convert(String source) { List<Object> list = new ArrayList<>(); JSONArray jsonArray = JSON.parseArray(source); for (int i = 0; i < jsonArray.size(); i++) { //解析rule属性 JSONObject jsonObject = (JSONObject) jsonArray.getJSONObject(i).get("rule"); Object object = JSON.toJavaObject(jsonObject, ruleClass); list.add(object); } return list; } }
从控制台发布配置到配置中心时,FlowRuleEntity转换为FlowRule。


浙公网安备 33010602011771号