hadoop_stack - 随笔分类 - papering

摘要：https://cwiki.apache.org/confluence/display/FLUME/Flume+NG+Performance+Measurements 阅读全文

posted @ 2019-06-05 09:39 papering 阅读(214) 评论(0) 推荐(0)

摘要：flume一开始是cloudlera的项目当时他们的工程师需要一次次地为客户编写工具来实现数据的自动化导入阅读全文

posted @ 2019-06-04 22:42 papering 阅读(318) 评论(0) 推荐(0)

摘要：http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html Feature Phase 1: Non-work-preserving RM restart As of Ha 阅读全文

posted @ 2019-05-31 20:44 papering 阅读(628) 评论(0) 推荐(0)

唯品会HDFS性能挑战和优化实践

摘要：唯品会HDFS性能挑战和优化实践原创：大数据平台唯技术 4月1日 https://mp.weixin.qq.com/s/LMa99ubgACI4eaDV3G-6gw 阅读全文

posted @ 2019-05-28 23:42 papering 阅读(293) 评论(0) 推荐(0)

构建Hadoop监控共同体

摘要：HDFS监控背后那些事儿，构建Hadoop监控共同体原创：应用研发部京东云 2018-12-19 https://mp.weixin.qq.com/s/kulwDgwu-rYf4SvQ1dOwcw 阅读全文

posted @ 2019-05-28 23:40 papering 阅读(186) 评论(0) 推荐(0)

消除单点故障 flume

摘要：p44 阅读全文

posted @ 2019-05-28 00:38 papering 阅读(472) 评论(0) 推荐(0)

hdfs ha

摘要：ha 阅读全文

posted @ 2019-05-28 00:24 papering 阅读(136) 评论(0) 推荐(0)

Put queue for MemoryTransaction of capacity 10000 full, consider committing more frequently, increasing capacity or increasing thread count flume capacity 时间数

摘要：[root@d log]# ps -aux | grep javaroot 550 0.2 1.9 6000924 641384 pts/1 Sl+ 17:39 0:13 /usr/java/jdk1.8.0_101/bin/java -Xmx20m -Dflume.root.logger=INFO 阅读全文

posted @ 2018-11-15 19:08 papering 阅读(529) 评论(0) 推荐(0)

引入 netty网关，向flume提交数据

摘要：netty 处理http请求阅读全文

posted @ 2018-11-13 09:28 papering 阅读(477) 评论(0) 推荐(0)

CSDN BI Flume

摘要：https://so.csdn.net/so/search/s.do?q=bytebuf&t=%20&u= 阅读全文

posted @ 2018-11-09 12:07 papering 阅读(248) 评论(0) 推荐(0)

node.js使用require给flume提交请求

摘要：node.js使用require给flume提交请求 - 简书 https://www.jianshu.com/p/02c20e2d011a 玄月府的小妖在debug 关注 2017.04.15 09:22* 最后编辑于 2017.12.06 17:44 字数 43 阅读 120评论 0喜欢 1 n 阅读全文

posted @ 2018-11-01 15:59 papering 阅读(265) 评论(0) 推荐(0)

flume sink两种类型 file_rool 自定义sing com.mycomm.MySink even if there is only one event, the event has to be sent in an array

摘要：mkdir /data/UnifiedLog/; cd /data/UnifiedLog/; wget http://mirror.bit.edu.cn/apache/flume/1.8.0/apache-flume-1.8.0-bin.tar.gz; tar -xvf apache-flume-1 阅读全文

posted @ 2018-10-30 16:42 papering 阅读(329) 评论(0) 推荐(0)

flume-ng script should first try finding java from PATH and then try using bigtop, instead of vice-versa

摘要：[FLUME-1154] flume-ng script should first try finding java from PATH and then try using bigtop, instead of vice-versa - ASF JIRA https://issues.apache 阅读全文

posted @ 2018-08-07 15:59 papering 阅读(162) 评论(0) 推荐(0)

pollable event-driven Flume source

摘要：2种source 阅读全文

posted @ 2018-08-03 22:13 papering 阅读(208) 评论(0) 推荐(0)

Space for commit to queue couldn't be acquired

摘要：18/07/27 16:53:53 ERROR source.ExecSource: Failed while running command: tail -F /home/MyBgJavaLan/pom.xmlorg.apache.flume.ChannelFullException: Space 阅读全文

posted @ 2018-07-27 17:02 papering 阅读(457) 评论(0) 推荐(0)

Flume-ng-sdk源码分析

摘要：Flume 实战(2)--Flume-ng-sdk源码分析 - mumuxinfei - 博客园 http://www.cnblogs.com/mumuxinfei/p/3823266.html 阅读全文

posted @ 2018-07-26 21:38 papering 阅读(181) 评论(0) 推荐(0)

将线上服务器生成的日志信息实时导入kafka，采用agent和collector分层传输，app的数据通过thrift传给agent，agent通过avro sink将数据发给collector，collector将数据汇集后，发送给kafka

摘要：记flume部署过程中遇到的问题以及解决方法（持续更新） - CSDN博客 https://blog.csdn.net/lijinqi1987/article/details/77449889 现将调试过程中遇到的问题以及解决方法记录如下： 1、 [ERROR - org.apache.thrift 阅读全文

posted @ 2018-07-26 20:52 papering 阅读(598) 评论(0) 推荐(0)

线程池最大线程数

摘要：一个flume agent异常的解决过程记录 - 施文涛 - 博客园 https://www.cnblogs.com/dongqingswt/p/5287778.html 一个flume agent异常的解决过程记录今天在使用flume agent的时候，遇到了一个异常，现把解决的过程记录如下：阅读全文

posted @ 2018-07-26 20:36 papering 阅读(2408) 评论(0) 推荐(0)

using the flume-ng-sql-source plugin to push data from Mysql DB to Spark

摘要：org.apache.flume.EventDeliveryException while running flume and sending data to spark · Issue #21 · keedio/flume-ng-sql-source https://github.com/keed 阅读全文

posted @ 2018-07-26 11:21 papering 阅读(215) 评论(0) 推荐(0)

monitor and move the log content to our big data system

摘要：Apache Flume HDFS Sink Tutorial | HowToProgram https://howtoprogram.xyz/2016/08/01/apache-flume-hdfs-sink-tutorial/ 阅读全文

posted @ 2018-07-26 10:03 papering 阅读(148) 评论(0) 推荐(0)

papering

随笔分类 - hadoop_stack