一、CountWindoow不分组,整体划分窗口
DataStream<String> lines = env.socketTextStream("192.168.87.130", 8888);
SingleOutputStreamOperator<Integer> word = lines.flatMap(new FlatMapFunction<String, Integer>() {
private static final long serialVersionUID = 1L;
@Override
public void flatMap(String value, Collector<Integer> out)
throws Exception {
out.collect(Integer.parseInt(value));
}
});
//不分组,整体当成一个组
AllWindowedStream<Integer, GlobalWindow> windowStream = word.countWindowAll(5);
SingleOutputStreamOperator<Integer> summed = windowStream.sum(0);
summed.print();
输入:
2 2 2 2 2
输入:
10
二、CountWindow先分组,再划分窗口
DataStream<String> lines = env.socketTextStream("localhost", 8888);
SingleOutputStreamOperator<Tuple2<String, Integer>> word = lines.flatMap(new FlatMapFunction<String, Tuple2<String, Integer>>() {
private static final long serialVersionUID = 7179469626039725354L;
@Override
public void flatMap(String value,
Collector<Tuple2<String, Integer>> out) throws Exception {
String[] splitStr = value.split(",");
out.collect(Tuple2.of(splitStr[0], Integer.parseInt(splitStr[1])));
}
});
//先分组,再划分窗口
KeyedStream<Tuple2<String, Integer>, Tuple> keyed = word.keyBy(0);
WindowedStream<Tuple2<String, Integer>, Tuple, GlobalWindow> windowStream = keyed.countWindow(5);
SingleOutputStreamOperator<Tuple2<String,Integer>> summed = windowStream.sum(1);
summed.print();
输入:
flink,2 spark,3 flink,1 flink,3 flink,4 flink,5
输出:
flink,15
三、Tumbling滚动窗口(不分组滚动时间窗口)
DataStream<String> lines = env.socketTextStream("192.168.87.130", 8888);
SingleOutputStreamOperator<Integer> word = lines.flatMap(new FlatMapFunction<String, Integer>() {
private static final long serialVersionUID = 1L;
@Override
public void flatMap(String value, Collector<Integer> out)
throws Exception {
out.collect(Integer.parseInt(value));
}
});
//不分组,整体当成一个组
AllWindowedStream<Integer, TimeWindow> windowStream = word.timeWindowAll(Time.seconds(5));
SingleOutputStreamOperator<Integer> summed = windowStream.sum(0);
summed.print();
5秒划分一个窗口,控制台输入等待5s后sum
输入:
1
1
输出:2
四、Tumbling Window(先分组,滚动窗口)
DataStream<String> lines = env.socketTextStream("192.168.87.130", 8888);
SingleOutputStreamOperator<Tuple2<String, Integer>> word = lines.flatMap(new FlatMapFunction<String, Tuple2<String, Integer>>() {
private static final long serialVersionUID = 7179469626039725354L;
@Override
public void flatMap(String value,
Collector<Tuple2<String, Integer>> out) throws Exception {
String[] splitStr = value.split(",");
out.collect(Tuple2.of(splitStr[0], Integer.parseInt(splitStr[1])));
}
});
//先分组,再划分窗口
KeyedStream<Tuple2<String, Integer>, Tuple> keyed = word.keyBy(0);
WindowedStream<Tuple2<String, Integer>, Tuple, TimeWindow> windowStream = keyed.timeWindow(Time.seconds(5));
SingleOutputStreamOperator<Tuple2<String,Integer>> summed = windowStream.sum(1);
summed.print();
输入:
abc,1
abc,2
输出:
abc,1
abc,2
每5s划分一个窗口,有多个组,5s后多个组都会划分窗口