fxjwind

摘要： 1 introduction 1.2 Asynchronous by design two most common ways to work with or implement an asynchronous API, 1.2.1 Callbacks 回调函数，本质， move the execution of these methods from the “caller” thr... 阅读全文

摘要： listeners, Listener List - Comma-separated list of URIs we will listen on and their protocols. Specify hostname as 0.0.0.0 to bind to all interfaces. Leave hostname empty to bind to default in... 阅读全文

摘要： NetworkEnvironment 是一个TaskManager对应一个，而不是一个task对应一个 其中最关键的是networkBufferPool， operator产生的中间结果，ResultPartition，或是input数据，InputGate 都是需要memory来暂存的，这就需要networkBufferPool来管理这部分内存 /** * Networ... 阅读全文

摘要： 参考， http://wuchong.me/blog/2016/05/25/flink-internals-window-mechanism/ http://wuchong.me/blog/2016/06/06/flink-internals-session-window/ WindowOperator window operator通过WindowAssigner和Tr... 阅读全文

摘要： org.apache.flink.streaming.api.windowing.triggers; Trigger public abstract class Trigger implements Serializable { /** * Called for every element that gets added to a pane. The resul... 阅读全文

摘要： 如果要考虑易用性和效率，使用rocksDB来替代普通内存的kv是有必要的 有了rocksdb，可以range查询，可以支持columnfamily，可以各种压缩 但是rocksdb本身是一个库，是跑在RocksDBStateBackend中的 所以taskmanager挂掉后，数据还是没了， 所以RocksDBStateBackend仍然需要类似HDFS这样的分布式存储来存储snapsho... 阅读全文

摘要： 在Flink – Checkpoint 没有描述了整个checkpoint的流程，但是对于如何生成snapshot和恢复snapshot的过程，并没有详细描述，这里补充 StreamOperator /** * Basic interface for stream operators. Implementers would implement one of * {@link ... 阅读全文

摘要： public class StreamTaskState implements Serializable, Closeable { private static final long serialVersionUID = 1L; private StateHandle operatorState; private StateHandle funct... 阅读全文

摘要： JobManager failover LeaderLatch private synchronized void setLeadership(boolean newValue){ boolean oldValue = hasLeadership.getAndSet(newValue); if ( oldValue && !newValue ) //原来是leader，... 阅读全文

摘要： Where did we come from? With the 0.9.0-milestone1 release, Apache Flink added an API to process relational data with SQL-like expressions called the Table API. The central concept of this API is a Ta... 阅读全文