Flume接收kafka消息过多导致的GC错误的解决办法
当flume接收kafka消息过多会导致如下错误:
(SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.OutOfMemoryError: GC overhead limit exceeded
报错原因:
因为flume的jvm设置的最大内存默认为20M,只需要改一下这个值即可,如下图所示:
vim flume/bin/flume-ng
参考:
https://blog.csdn.net/sunxiaoju/article/details/105694277?utm_medium=distribute.pc_relevant.none-task-blog-2~default~baidujs_title~default-0.pc_relevant_default&spm=1001.2101.3001.4242.1&utm_relevant_index=3