Flink1.10.0编译hadoop2.7.2 编译flink-shaded-hadoop-2-uber

Flink1.10.0编译hadoop2.7.2 编译flink-shaded-hadoop-2-uber

从Flink 1.11开始，flink-shaded-hadoop-2-uberFlink项目不再正式支持使用发行版。
如果想建立flink-shaded对供应商特定的Hadoop版本，您必须首先描述配置特定供应商的Maven仓库在本地Maven安装在这里。
运行以下命令以flink-shaded针对所需的Hadoop版本（例如对于version 2.6.5-custom）进行构建和安装：

mvn clean install -Dhadoop.version=2.6.5-custom

1. 下载/flink-shaded

前往git 下载/flink-shaded 下载relese-10.0 因为10.0是包含hadoop-bin的，1.11 不包含hadoop

https://github.com/apache/flink-shaded.git

2. 解压至linux

tar -zxvf flink-shaded-release-10.0.tar.gz

3. 进入flink-shaded-hadoop-2-parent，修改pom

tar -zxvf flink-shaded-release-10.0.tar.gz

修改flink-shaded-hadoop-2-uber下pom.xml文件, 执行zookeeper的版本

[ryxiong@node01 flink-shaded-hadoop-2-parent]$ cd flink-shaded-hadoop-2-uber/
[ryxiong@node01 flink-shaded-hadoop-2-uber]$ vim pom.xml 

4. 进入flink-shaded-release-10.0下 修改pom

[ryxiong@node01 job]$ cd flink-shaded-release-10.0/
[ryxiong@node01 flink-shaded-release-10.0]$ vim pom.xml

在profiles标签中添加如下内容

 <profile>
    <id>vendor-repos</id>
    <activation>
        <property>
            <name>vendor-repos</name>
        </property>
    </activation>   <!-- Add vendor maven repositories -->
    <repositories>       <!-- Cloudera -->
        <repository>
            <id>cloudera-releases</id>
            <url>https://maven.aliyun.com/repository/central</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>
</profile>

添加阿里云镜像, 编译过程中下载块

5. 配置完毕，运行前提是需要linux配置maven

在flink-shaded-release-10.0执行

mvn clean install -DskipTests -Dhadoop.version=2.7.2 -Drat.skip=true

进入漫长的编译过程, 没有ERROR即成功

[INFO] Reactor Summary:
[INFO]
[INFO] flink-shaded ...................................... SUCCESS [1.266s]
[INFO] flink-shaded-force-shading ........................ SUCCESS [0.383s]
[INFO] flink-shaded-asm-7 ................................ SUCCESS [0.464s]
[INFO] flink-shaded-guava-18 ............................. SUCCESS [0.882s]
[INFO] flink-shaded-netty-4 .............................. SUCCESS [4.089s]
[INFO] flink-shaded-netty-tcnative-dynamic ............... SUCCESS [0.457s]
[INFO] flink-shaded-jackson-parent ....................... SUCCESS [0.015s]
[INFO] flink-shaded-jackson-2 ............................ SUCCESS [0.816s]
[INFO] flink-shaded-jackson-module-jsonSchema-2 .......... SUCCESS [0.595s]
[INFO] flink-shaded-hadoop-2-parent ...................... SUCCESS [0.017s]
[INFO] flink-shaded-hadoop-2 ............................. SUCCESS [8.014s]
[INFO] flink-shaded-hadoop-2-uber ........................ SUCCESS [10.657s]
[INFO] flink-shaded-zookeeper-parent ..................... SUCCESS [0.011s]
[INFO] flink-shaded-zookeeper-3.4 ........................ SUCCESS [41.310s]
[INFO] flink-shaded-zookeeper-3.5 ........................ SUCCESS [33.234s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:42.440s
[INFO] Finished at: Fri Apr 09 16:56:24 CST 2021
[INFO] Final Memory: 45M/1112M
[INFO] ------------------------------------------------------------------------
[mac@ryxiong flink-shaded-release-10.0]$ll

编译好的路径

[mac@ryxiong target]$pwd
~/flink-shaded-release-10.0/flink-shaded-hadoop-2-parent/flink-shaded-hadoop-2-uber/target

6. 编译好的jar包 放入flinke1.10.0的lib下

[ryxiong@node01 flink-shaded-release-10.0]$ cp flink-shaded-hadoop-2-uber-2.7.2-10.0.jar /opt/module/flink-1.10.0/lib/

 

本人编译好的链接：https://pan.baidu.com/s/14ZBKOiJH6NxLtaazccE0OA

提取码：d4el