模拟flink处理无限数据流 - 指南

如果没有在 linux 环境下安装 flink ，先看我的上一篇文章：如何搭建Linux环境下的flink本地集群-CSDN博客

使用工具：IntelliJ IDEA 2021，Maven 3.6.1

第一步，创建一个空的 Maven 项目，导入依赖

4.0.0
org.example
flinkLearn
1.0-SNAPSHOT
org.apache.flink
flink-runtime-web
${flink.version}
ch.qos.logback
logback-classic
1.2.11
org.apache.flink
flink-streaming-java
${flink.version}
org.apache.flink
flink-clients
${flink.version}
8
8
1.17.0
org.apache.maven.plugins
maven-shade-plugin
3.2.4
package
shade
com.google.code.findbugs:jsr305
org.slf4j:*
log4j:*
*:*
META-INF/*.SF
META-INF/*.DSA
META-INF/*.RSA

等等依赖下载完毕

第二步，编写分词处理无界流代码

此刻先不要运行，因为还没有数据源

import org.apache.flink.api.common.typeinfo.Types;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.util.Collector;
public class StreamNoBundedWordCount {
public static void main(String[] args) throws Exception{
//拿到执行环境
StreamExecutionEnvironment evn = StreamExecutionEnvironment.getExecutionEnvironment();
//数据源 source
//注：加 returns 是因为 lambda 表达式无法识别二元组中的类型，故手动指定以避免报错
DataStreamSource source = evn.socketTextStream("localhost", 8888);
SingleOutputStreamOperator> sum = source.flatMap((String value, Collector> out) -> {
String[] words = value.split(" ");
for (String word : words) {
out.collect(Tuple2.of(word, 1));
}
})
.returns(Types.TUPLE(Types.STRING,Types.INT))
.keyBy((Tuple2 value) -> value.f0)
.sum(1);
//写入,sink
sum.print();
//执行
evn.execute();
}
}