随笔分类 -  flink

1 2 下一页
flink-sql解析canal-json实现实时同步
摘要：package com.lezhi.business.dxxbs.transmission.table import com.lezhi.common.{CommonTransmissonFunciton, SystemParams} import org.apache.flink.streamin 阅读全文
posted @ 2021-07-12 11:30 nohert 阅读(1661) 评论(0) 推荐(0)
flink爬坑记录一
摘要：问题一 使用命令提交flink任务 flink run -c com.lezhi.business.dxxbs.transmission.ExecuteDML /data/jar/gkt-bigData-flink-1.0-SNAPSHOT-jar-with-dependencies.jar --i 阅读全文
posted @ 2021-07-09 15:02 nohert 阅读(3904) 评论(0) 推荐(0)
flink run参数
摘要：flink actions are "run", "list", "info", "savepoint", "stop", or "cancel".Specify the version option (-v or --version) to print Flink version. Specify 阅读全文
posted @ 2021-07-07 19:00 nohert 阅读(1483) 评论(0) 推荐(0)
flink-Stream解析canal-json数据
摘要：引入依赖 <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.33</version> </dependency> val env = StreamExecutionEn 阅读全文
posted @ 2021-06-24 19:34 nohert 阅读(908) 评论(0) 推荐(0)
flink 自定义ProcessFunction方法
摘要：import it.bigdata.flink.study.SensorReding import org.apache.flink.api.common.state.{ValueState, ValueStateDescriptor} import org.apache.flink.configu 阅读全文
posted @ 2021-06-24 19:32 nohert 阅读(404) 评论(0) 推荐(0)
flink下沉数据到kafka
摘要：import it.bigdata.flink.study.SensorReding import org.apache.flink.api.common.serialization.SimpleStringSchema import org.apache.flink.streaming.api.s 阅读全文
posted @ 2021-06-24 19:31 nohert 阅读(102) 评论(0) 推荐(0)
flink下沉数据到mysql
摘要：import java.sql.{Connection, Driver, DriverManager, PreparedStatement}import it.bigdata.flink.study.SensorRedingimport org.apache.flink.configuration. 阅读全文
posted @ 2021-06-24 19:29 nohert 阅读(152) 评论(0) 推荐(0)
flink jdbc方式下沉到hive
摘要：import java.sql.{Connection, DriverManager, PreparedStatement} import it.bigdata.flink.study.SensorReding import org.apache.flink.configuration.Config 阅读全文
posted @ 2021-06-24 19:28 nohert 阅读(663) 评论(0) 推荐(0)
flink Sink file
摘要：import it.bigdata.flink.study.SensorReding import org.apache.flink.api.common.serialization.SimpleStringEncoder import org.apache.flink.core.fs.Path i 阅读全文
posted @ 2021-06-24 19:12 nohert 阅读(120) 评论(0) 推荐(0)
flinkSinkES
摘要：import java.util import it.bigdata.flink.study.SensorReding import org.apache.flink.api.common.functions.RuntimeContext import org.apache.flink.stream 阅读全文
posted @ 2021-06-24 19:11 nohert 阅读(313) 评论(0) 推荐(0)
flink自定义Map方法
摘要：class MyRichMapper extends RichMapFunction[SensorReding,String]{ override def open(parameters: Configuration): Unit = { //做一些初始化操作，比如数据库的连接 // getRunt 阅读全文
posted @ 2021-06-24 19:06 nohert 阅读(408) 评论(0) 推荐(0)
flink 自定义filter
摘要：数据 sensor_1,1547718101,35.8 sensor_1,1547718102,22.2 sensor_1,1547718101,55.3 sensor_1,1547718102,24.1 sensor_1,1547718103,57 sensor_1,1547718103,58 s 阅读全文
posted @ 2021-06-24 19:03 nohert 阅读(380) 评论(0) 推荐(0)
flink分流合流
摘要：数据： sensor_1,1547718101,35.8 sensor_1,1547718102,22.2 sensor_1,1547718101,55.3 sensor_1,1547718102,24.1 sensor_1,1547718103,57 sensor_1,1547718103,58 阅读全文
posted @ 2021-06-24 18:56 nohert 阅读(187) 评论(0) 推荐(0)
flnk算子reduce的简单使用和自定义reduce
摘要：import org.apache.flink.api.common.functions.{FilterFunction, ReduceFunction, RichMapFunction} import org.apache.flink.configuration.Configuration imp 阅读全文
posted @ 2021-06-24 18:52 nohert 阅读(275) 评论(0) 推荐(0)
flink分组取最小值
摘要：import org.apache.flink.api.common.functions.{FilterFunction, ReduceFunction, RichMapFunction} import org.apache.flink.configuration.Configuration imp 阅读全文
posted @ 2021-06-24 18:49 nohert 阅读(255) 评论(0) 推荐(0)
flink批处理第一个代码WordCount
摘要：import org.apache.flink.api.scala.ExecutionEnvironment import org.apache.flink.api.scala._ object WordCount { /** * 第一个flink，wordcount * @param args * 阅读全文
posted @ 2021-06-24 18:46 nohert 阅读(73) 评论(0) 推荐(0)
FlinkStream第一个代码WordCount
摘要：import org.apache.flink.api.java.utils.ParameterTool import org.apache.flink.streaming.api.scala.{DataStream, StreamExecutionEnvironment} import org.a 阅读全文
posted @ 2021-06-24 18:44 nohert 阅读(52) 评论(0) 推荐(0)
flink定义各类source
摘要：package it.bigdata.flink.study import java.util.Properties import org.apache.flink.api.common.serialization.SimpleStringSchema import org.apache.flink 阅读全文
posted @ 2021-06-24 18:42 nohert 阅读(94) 评论(0) 推荐(0)
flink个人基础笔记
摘要：flink代码开发基础随记 1.常用设置 设置共享组：slotsharingGroup(“a”) 关闭合并任务链操作，每个任务独享solt：env.disableOperatorChaining() 设置并行度： env.setParallelism(1) Flatmap用法： FlatMap(Li 阅读全文
posted @ 2021-06-24 18:39 nohert 阅读(254) 评论(0) 推荐(0)
flink任务提交流程
摘要：Flink运行时组件： JobManager 作业管理器 TaskManager 任务管理器 ResourceManager 资源管理器 Dispacher 分发器 阅读全文
posted @ 2021-06-24 18:37 nohert 阅读(180) 评论(0) 推荐(0)

1 2 下一页