fxjwind

摘要： WebRuntimeMonitor .GET("/jobs/:jobid/vertices/:vertexid/metrics", handler(new JobVertexMetricsHandler(metricFetcher))) .GET("/jobs/:jobid/metrics", handler(new JobMetricsHandler(metricFetcher))) ... 阅读全文

摘要： http://commons.apache.org/proper/commons-math/userguide/stat.html mark DescriptiveStatistics maintains the input data in memory and has the capability of producing "rolling" statistics computed f... 阅读全文

摘要： Jobmanager的submitJob逻辑， /** * Submits a job to the job manager. The job is registered at the libraryCacheManager which * creates the job's class loader. The job graph is appended to the corr... 阅读全文

摘要： Flink 自己创建一套独立的类型系统， 参考， https://ci.apache.org/projects/flink/flink-docs-release-0.10/internals/types_serialization.html 为何要自己搞一套，而不像其他的平台一样让编程语言或serialization framework来天然做掉？ Flink tries to know a... 阅读全文

摘要： kafka 0.10.0.0 released Interceptors的概念应该来自flume 参考，http://blog.csdn.net/xiao_jun_0820/article/details/38111305 比如，flume提供的 Timestamp Interceptor Host Interceptor Static Interceptor Regex ... 阅读全文

摘要： 关于我们为什么需要Schema Registry？ 参考， https://www.confluent.io/blog/how-i-learned-to-stop-worrying-and-love-the-schema-part-1/ https://www.confluent.io/blog/schema-registry-kafka-stream-processing-yes-virg... 阅读全文

摘要： 32， 33都是和时间相关的， KIP-32 - Add timestamps to Kafka message 引入版本，0.10.0.0 需要给kafka的message加上时间戳，这样更方便一些， 比如在做retention，rolling，或getMessageByTime的时候 在config里面可以配置， message.timestamp.type - This to... 阅读全文

摘要： Flink resource manager的作用如图， FlinkResourceManager /** * * Worker allocation steps * * * The resource manager decides to request more workers. This can happen in order * t... 阅读全文

摘要： InstanceManager用于管理JobManager申请到的taskManager和slots资源 /** * Simple manager that keeps track of which TaskManager are available and alive. */ public class InstanceManager { // ---------------... 阅读全文

摘要： F1: A Distributed SQL Database That Scales http://disksing.com/understanding-f1-schema-change mark 阅读全文