Flume接收kafka消息过多导致的GC错误的解决办法

flume接收kafka消息过多会导致如下错误:

(SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.OutOfMemoryError: GC overhead limit exceeded

  

 

 报错原因:

因为flume的jvm设置的最大内存默认为20M,只需要改一下这个值即可,如下图所示:

vim flume/bin/flume-ng

 

 

 

 

 

参考:

https://blog.csdn.net/sunxiaoju/article/details/105694277?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_title~default-0.pc_relevant_default&spm=1001.2101.3001.4242.1&utm_relevant_index=3

posted @ 2022-03-11 17:55  Geoffrey-Peng  阅读(414)  评论(0)    收藏  举报