随笔分类 -  Flume

flume interceptors flume拦截器
摘要：flume用户自定义拦截器.创建flume-demo的maven项目. 创建项目文件POM.xml. <dependency> <groupId>org.apache.flume</groupId> <artifactId>flume-ng-core</artifactId> <version>1. 阅读全文
posted @ 2021-03-09 10:18 kpwong 阅读(132) 评论(0) 推荐(0)
Flume Agent原理
摘要： 阅读全文
posted @ 2021-03-07 11:55 kpwong 阅读(63) 评论(0) 推荐(0)
flume事务机制
摘要：两个事务: Put事务，take事务 阅读全文
posted @ 2021-03-07 10:31 kpwong 阅读(186) 评论(0) 推荐(0)
生成环境 经常使用 taildir : 支持断点续传
摘要：配置文件 flume-taildir-logger.conf： # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1. 阅读全文
posted @ 2021-03-07 10:09 kpwong 阅读(229) 评论(0) 推荐(0)
dir-flume-hdfs
摘要：配置文件flume-dir-hdfs.conf a3.sources = r3 a3.sinks = k3 a3.channels = c3 # Describe/configure the source a3.sources.r3.type = spooldir a3.sources.r3.spo 阅读全文
posted @ 2021-03-06 16:55 kpwong 阅读(73) 评论(0) 推荐(0)
logfile-flume-hdfs
摘要：conf配置文件flume-file-hdfs.conf # Name the components on this agent a2.sources = r2 a2.sinks = k2 a2.channels = c2 # Describe/configure the source a2.sou 阅读全文
posted @ 2021-03-06 16:29 kpwong 阅读(206) 评论(0) 推荐(0)
netcat-flume-logger
摘要：flume-netcat-logger.conf 文件编写: # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.s 阅读全文
posted @ 2021-03-06 10:24 kpwong 阅读(128) 评论(0) 推荐(0)
Flume与kafka集成
摘要：1）配置flume(flume-kafka.conf) # define a1.sources = r1 a1.sinks = k1 a1.channels = c1 # source a1.sources.r1.type = exec a1.sources.r1.command = tail -F 阅读全文
posted @ 2020-11-25 09:04 kpwong 阅读(164) 评论(0) 推荐(0)