flume学习(三)---sink
本文根据flume官网提供的Flume 1.8.0 User Guide整理得出的,主要内容是总结了flume1.8.0支持的各种sink,见下表。
| sink名称 | type | 数据目的地 | 工作方式 | 备注 |
| HDFS Sink | hdfs | HDFS | ||
| Hive Sink | hive | hive | ||
| Logger Sink | logger | 自定义 | 日志级别为INFO,主要用于调试 | |
| Avro Sink | avro | 外部avro客户端 |
将接收到的events转换成avro events 并发送给指定地址的avro客户端 |
|
| Thrift Sink | thrift | 外部thrift客户端 |
将接收到的events转换成thrift events 并发送给指定地址的avro客户端 |
|
| IRC Sink | irc | IRC目的地 | ||
| File Roll Sink | file_roll | 本地文件系统 | ||
| Null Sink | null | 无 | 丢弃所有数据 | |
| Hbase Sink | hbase | Hbase | 这两个sink同属一个大类:HbaseSinks | |
| AsyncHbaseSink | asynchbase | Hbase(异步模式) | ||
| MorphlineSolrSink |
org.apache.flume. sink.solr.morphline. MorphlineSolrSink |
Apache Solr servers |
This sink extracts data from Flume events, transforms it, and loads it in near-real-time into Apache Solr servers, which in turn serve queries to end users or search applications. |
|
| ElasticSearchSink |
org.apache.flume. sink.elasticsearch. ElasticSearchSink |
elasticsearch cluster | ||
| Kite Dataset Sink |
org.apache.flume. sink.kite.DatasetSink |
Kite Dataset | ||
| Kafka Sink |
org.apache.flume. sink.kafka.KafkaSink |
Kafka topic | ||
| HTTP Sink |
http |
一个URI地址(POST) | ||
| Custom Sink |
自定义FQCN |
自定义 |
浙公网安备 33010602011771号