FLINK-状态管理-状态使用
1.ValueState:
private var lastTempState: ValueState[Double] = _ override def open(parameters: Configuration): Unit = { val lastTempDescriptor = new ValueStateDescriptor[Double]("lastTemp", classOf[Double]) lastTempState = getRuntimeContext.getState[Double](lastTempDescriptor) }
2.ListState:
private var itemState : ListState[ItemViewCount] = _ override def open(parameters: Configuration): Unit = { //命名状态变量的名字和类型 val itemStateDescription: ListStateDescriptor[ItemViewCount] = new ListStateDescriptor[ItemViewCount]("itemState", classOf[ItemViewCount]) itemState = getRuntimeContext.getListState(itemStateDescription) }
Flink提供了三种基于key/value的state接口,ListState接口适用于缓存多个值的计算。具体实现之前,因为state必须是基于key,且必须获取getRuntimeContext,state必须同时满足两个条件:
- 直接基于keyedStream或者由keyedStream转换来的windowedStream
- 必须继承RichFunction
实际实现时候,因为windowedStream在scala中不能实现RichWindowFunction,因此在main中使用flatmap间接实现了windowFunction中的功能:
val fromTransactionDataStream = watermarkTransaction .keyBy(_.code) .window(TumblingEventTimeWindows.of(Time.seconds(10))) val transaction = fromTransactionDataStream .apply(new StockTransactionApply) .keyBy(_._3) .flatMap(new TransactionStateFlatMapFunction)
3.MapState
private var behaviorMapState: MapState[String, Int] = _ override def open(parameters: Configuration): Unit = { // 创建StateDescriptor val behaviorMapStateDescriptor = new MapStateDescriptor[String, Int]("behaviorMap", classOf[String], classOf[Int]) // 通过StateDescriptor获取运行时上下文中的状态 behaviorMapState = getRuntimeContext.getMapState(behaviorMapStateDescriptor) }
4.Reducing和AggregatingState
ReducingState和AggregatingState在注册StateDescriptor时,还需要实现一个ReduceFunction或AggregationFunction。下面的代码注册ReducingStateDescriptor时实现一个YourReduceFunction,YourReduceFunction实现了ReduceFunction。我们在ReducingState中使用add(in: T)方法向状态里增加一个元素,新元素和状态中已有数据通过ReduceFunction两两聚合。AggregatingState的使用方法与之类似。
val reducingStateDescriptor = new ReducingStateDescriptor[UserBehavior]("reducing", new YourReduceFunction, classOf[UserBehavior])

浙公网安备 33010602011771号