Flink监控信息写入到PushGateway出现 java.io.IOException: Response code from http xx was 200问题

最近在将Flink的Metrics存储由InfluxDB迁移到Prometheus的时候,由于使用的版本问题,导致一直出现下面的WARN日志

2020-05-16 12:36:57.794 [Flink-MetricRegistry-thread-1] WARN  o.a.flink.metrics.prometheus.PrometheusPushGatewayReporter  - Failed to push metrics to PushGateway with jobName flinkb70c1c134b3ba0a918a7b21a4d443982.
java.io.IOException: Response code from http://xx.xx.xx.xx:9091/metrics/job/flinkb70c1c134b3ba0a918a7b21a4d443982 was 200
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway.doRequest(PushGateway.java:297)
    at org.apache.flink.shaded.io.prometheus.client.exporter.PushGateway.push(PushGateway.java:105)
    at org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter.report(PrometheusPushGatewayReporter.java:76)
    at org.apache.flink.runtime.metrics.MetricRegistryImpl$ReporterTask.run(MetricRegistryImpl.java:430)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

这里使用的PushGateway版本是1.2.0,Flink1.10.0源码使用Prometheus的版本还是0.3.0,最后发现是因为版本不兼容的问题,把PushGateway的版本降到0.9.1就可以了

 

  

posted @ 2020-05-22 16:35  博而不客  阅读(2647)  评论(0编辑  收藏  举报