文章分类 - 大数据
摘要：1.1. Azure Data Lake Store The Azure Data Lake Store destination writes data to the Microsoft Azure Data Lake Store. You can use the Azure Data Lake S
阅读全文
摘要：1.1. Amazon S3 The Amazon S3 destination writes data to Amazon S3. To write data to an Amazon Kinesis Firehose delivery system, use the Kinesis Fireho
阅读全文
摘要：1.1. Destinations A destination stage represents the target for a pipeline. You can use one or more destinations in a pipeline. You can use different
阅读全文
摘要：1.1. File Tail The File Tail origin reads lines of data as they are written to an active file after reading related archived files in the same directo
阅读全文
摘要：1.1. Directory The Directory origin reads data from files in a directory. The origin can use multiple threads to enable the parallel processing of fil
阅读全文
摘要：1.1. Amazon S3 The Amazon S3 origin reads objects stored in Amazon S3. The object names must share a prefix pattern and should be fully written. To re
阅读全文
摘要：1.1. Origins An origin stage represents the source for the pipeline. You can use a single origin stage in a pipeline. You can use different origins ba
阅读全文
摘要：1. Data Formats 1.1. Data Formats Overview Data formats - such as Avro, JSON, and log - are methods to encode data that adhere to generally accepted s
阅读全文
摘要：1.1. Pipeline Designer UI The following image shows the Pipeline Designer UI when you configure a pipeline: AREA/ICON NAME DESCRIPTION 1 Pipeline canv
阅读全文
摘要：1. 管道概念和设计 1.1. 设计数据流 你能在 pipeline 中分支或者合并一个数据流. 1.1.1. 数据流分叉 When you connect a stage to multiple stages, all data passes to all connected stages. Yo
阅读全文