Window Function
Window Function在窗口触发后,负责对窗口内的元素进行计算。Window Function分为两类: 增量聚合和全量聚合。
增量聚合: 窗口不维护原始数据,只维护中间结果,每次基于中间结果和增量数据进行聚合。如: ReduceFunction、AggregateFunction。
全量聚合: 窗口需要维护全部原始数据,窗口触发进行全量聚合。如: ProcessWindowFunction。
AggregateFunction
有三个参数:输入类型(IN)、累加器类型(ACC)和输出类型(OUT),输入类型是输入流中的元素类型,AggregateFunction有一个add方法可以将一个输入元素添加到一个累加器中。该接口还具有创建初始累加器(createAccumulator方法)、将两个累加器合并到一个累加器(merge方法)以及从累加器中提取输出(类型为OUT)的方法。
public class AggFunctionOnWindow { public static final Tuple3[] MATH = new Tuple3[]{ Tuple3.of("class1", "jack", 100), Tuple3.of("class1", "Lucy", 80), Tuple3.of("class1", "jim", 90), Tuple3.of("class1", "tom", 110), Tuple3.of("class2", "wang", 90), Tuple3.of("class2", "li", 80), Tuple3.of("class2", "li2", 80), }; public static void main(String[] args) throws Exception { StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); DataStream<Tuple3<String, String, Integer>> input = env.fromElements(MATH); DataStream<Double> avgScore = input.keyBy(0).countWindow(3).aggregate(new AverageAggrate()); avgScore.print(); env.execute("aggFunction"); } public static class AverageAggrate implements AggregateFunction<Tuple3<String, String, Integer>, Tuple2<Integer, Integer>, Double> { /** * 创建累加器保存中间状态(sum,count) * * @return */ @Override public Tuple2<Integer, Integer> createAccumulator() { return Tuple2.of(0, 0); } @Override public Tuple2<Integer, Integer> add(Tuple3<String, String, Integer> value, Tuple2<Integer, Integer> accumulator) { //acc.f0 总成绩 //value.f2 表示成绩 //acc.f1 人数 System.out.println(accumulator.f0 + " --- " + value.f2); return new Tuple2<>(accumulator.f0 + value.f2, accumulator.f1 + 1); } @Override public Double getResult(Tuple2<Integer, Integer> accumulator) { return ((double) accumulator.f0) / accumulator.f1; } @Override public Tuple2<Integer, Integer> merge(Tuple2<Integer, Integer> a, Tuple2<Integer, Integer> b) { System.out.println(a.f0 + " " + b.f0 + " " + a.f1 + " " + b.f1); return new Tuple2<>(a.f0 + b.f0, a.f1 + b.f1); } } }
立志如山 静心求实
浙公网安备 33010602011771号