Flink1.9.0集成hadoop-cdh5.16.1编译

实验环境：

Centos7

jdk1.8

maven 3.6.1

flink1.9

hadoop2.6.0-cdh5.16.1

 

一、将本地的maven仓库setting中的mirror都删除掉

二、编译flink-shade源码

1、下载

wget https://archive.apache.org/dist/flink/flink-shaded-7.0/flink-shaded-7.0-src.tgz

 

2、修改项目pom

在项目的pom中加入如下仓库

<repositories>
<repository>
     <id>cloudera</id>
     <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
</repositories>

 

3、编译

mvn clean install -DskipTests -Dhadoop.version=2.6.0-cdh5.16.1

 

三、编译flink源码

1、下载源码

直接从Git上clone即可

git clone https://github.com/apache/flink.git

 

2、切换到相应的分支

git checkout release-1.9.0

 

3、编译

/opt/maven/apache-maven-3.6.1/bin/mvn clean install -DskipTests  -Pvendor-repos -Dhadoop.version=2.6.0-cdh5.16.1 -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Pinclude-hadoop

 

编译完成后有如下错误flink-metrics-influxdb

 

编译成功,

但是启动进程出现错误

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder

 

修改shade依赖

<relocation>
    <pattern>org.apache.commons.cli</pattern>
    <shadedPattern>org.shaded.commons.cli</shadedPattern>
</relocation>

 

编译后成功运行