Window Function

  Window Function在窗口触发后,负责对窗口内的元素进行计算。Window Function分为两类: 增量聚合和全量聚合。

  增量聚合: 窗口不维护原始数据,只维护中间结果,每次基于中间结果和增量数据进行聚合。如: ReduceFunction、AggregateFunction。
  全量聚合: 窗口需要维护全部原始数据,窗口触发进行全量聚合。如: ProcessWindowFunction。

AggregateFunction

  有三个参数:输入类型(IN)、累加器类型(ACC)和输出类型(OUT),输入类型是输入流中的元素类型,AggregateFunction有一个add方法可以将一个输入元素添加到一个累加器中。该接口还具有创建初始累加器(createAccumulator方法)、将两个累加器合并到一个累加器(merge方法)以及从累加器中提取输出(类型为OUT)的方法。

public class AggFunctionOnWindow {

    public static final Tuple3[] MATH = new Tuple3[]{
            Tuple3.of("class1", "jack", 100),
            Tuple3.of("class1", "Lucy", 80),
            Tuple3.of("class1", "jim", 90),
            Tuple3.of("class1", "tom", 110),
            Tuple3.of("class2", "wang", 90),
            Tuple3.of("class2", "li", 80),
            Tuple3.of("class2", "li2", 80),
    };

    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        DataStream<Tuple3<String, String, Integer>> input = env.fromElements(MATH);
        DataStream<Double> avgScore = input.keyBy(0).countWindow(3).aggregate(new AverageAggrate());
        avgScore.print();
        env.execute("aggFunction");
    }

    public static class AverageAggrate implements AggregateFunction<Tuple3<String, String, Integer>, Tuple2<Integer, Integer>, Double> {

        /**
         * 创建累加器保存中间状态(sum,count)
         *
         * @return
         */
        @Override
        public Tuple2<Integer, Integer> createAccumulator() {
            return Tuple2.of(0, 0);
        }

        @Override
        public Tuple2<Integer, Integer> add(Tuple3<String, String, Integer> value, Tuple2<Integer, Integer> accumulator) {
            //acc.f0 总成绩
            //value.f2 表示成绩
            //acc.f1 人数
            System.out.println(accumulator.f0 + " --- " + value.f2);
            return new Tuple2<>(accumulator.f0 + value.f2, accumulator.f1 + 1);
        }

        @Override
        public Double getResult(Tuple2<Integer, Integer> accumulator) {
            return ((double) accumulator.f0) / accumulator.f1;
        }

        @Override
        public Tuple2<Integer, Integer> merge(Tuple2<Integer, Integer> a, Tuple2<Integer, Integer> b) {
            System.out.println(a.f0 + "  " + b.f0 + "  " + a.f1 + " " + b.f1);
            return new Tuple2<>(a.f0 + b.f0, a.f1 + b.f1);
        }
    }
}

 

posted on 2022-05-01 23:07  溪水静幽  阅读(97)  评论(0)    收藏  举报