百度 Apollo 自定义安装第三方库（以 libtorch 为例）_apollo 使用自定义库

百度 Apollo 是一个非常优秀的自动驾驶框架，但我们平时在开发中也会遇到各种原 repo 没有处理的问题。笔者近期想用 pytorch 的 C++ 前端推理模型，但是遇到了 libtorch 版本与 pytorch 版本不匹配的问题，因此想自己安装一个新版本的 libtorch。

首先找到 Apollo 的 docker 是怎么安装 libtorch 或其他第三方库的。在 / apollo/docker/build/installers 文件夹下面，可以看到非常多的 install_***.sh，这些就是安装第三方库的脚本。原始的 install_libtorch.sh 如下

set -e
 
CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
. ${CURR_DIR}/installer_base.sh
 
# TODO(build): Docs on how to build libtorch on Jetson boards
# References:
#   https://github.com/ApolloAuto/apollo/blob/pre6/docker/build/installers/install_libtorch.sh
#   https://github.com/dusty-nv/jetson-containers/blob/master/Dockerfile.pytorch
#   https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-6-0-now-available
#   https://github.com/pytorch/pytorch/blob/master/docker/caffe2/ubuntu-16.04-cpu-all-options/Dockerfile
bash ${CURR_DIR}/install_mkl.sh
 
TARGET_ARCH="$(uname -m)"
 
##============================================================##
# libtorch_cpu
 
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    # https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.5.0%2Bcpu.zip
    VERSION="1.7.0-2"
    CHECKSUM="02fd4f30e97ce8911ef933d0516660892392e95e6768b50f591f4727f6224390"
elif [[ "${TARGET_ARCH}" == "aarch64" ]]; then
    VERSION="1.6.0-1"
    CHECKSUM="6d1fba522e746213c209fbf6275fa6bac68e360bcd11cbd4d3bdbddb657bee82"
else
    error "libtorch for ${TARGET_ARCH} not ready. Exiting..."
    exit 1
fi
 
PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"
DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_cpu
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_cpu ${VERSION}"
 
##============================================================##
# libtorch_gpu
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    VERSION="1.7.0-2"
    CHECKSUM="b64977ca4a13ab41599bac8a846e8782c67ded8d562fdf437f0e606cd5a3b588"
    PKG_NAME="libtorch_gpu-${VERSION}-cu111-linux-x86_64.tar.gz"
else # AArch64
    VERSION="1.6.0-1"
    PKG_NAME="libtorch_gpu-1.6.0-1-linux-aarch64.tar.gz"
    CHECKSUM="eeb5a223d9dbe40fe96f16e6711c49a3777cea2c0a8da2445d63e117fdad0385"
fi
 
DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_gpu
 
# Cleanup
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_gpu ${VERSION}"

可以看到这一段

PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"

DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"

download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"

tar xzf "${PKG_NAME}"
mv "${PKG_NAME%.tar.gz}" /usr/local/libtorch_cpu
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_cpu ${VERSION}"

实际上就是找到 Apollo 指定版本的 libtorch 然后交给 Apollo 编译，因此想要替换版本的话只需要自己把这几句下载解压之类的东西替换就行。

接下来看笔者自己写的新的 install_libtorch_new.sh：

set -e
 
CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
. ${CURR_DIR}/installer_base.sh
 
# TODO(build): Docs on how to build libtorch on Jetson boards
# References:
#   https://github.com/ApolloAuto/apollo/blob/pre6/docker/build/installers/install_libtorch.sh
#   https://github.com/dusty-nv/jetson-containers/blob/master/Dockerfile.pytorch
#   https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-6-0-now-available
#   https://github.com/pytorch/pytorch/blob/master/docker/caffe2/ubuntu-16.04-cpu-all-options/Dockerfile
bash ${CURR_DIR}/install_mkl.sh
 
TARGET_ARCH="$(uname -m)"
 
##============================================================##
# libtorch_cpu
 
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    # https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.5.0%2Bcpu.zip
    VERSION="1.7.0-2"
    CHECKSUM="02fd4f30e97ce8911ef933d0516660892392e95e6768b50f591f4727f6224390"
elif [[ "${TARGET_ARCH}" == "aarch64" ]]; then
    VERSION="1.6.0-1"
    CHECKSUM="6d1fba522e746213c209fbf6275fa6bac68e360bcd11cbd4d3bdbddb657bee82"
else
    error "libtorch for ${TARGET_ARCH} not ready. Exiting..."
    exit 1
fi
 
# PKG_NAME="libtorch_cpu-${VERSION}-linux-${TARGET_ARCH}.tar.gz"
# DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
# download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
PKG_NAME="/apollo/libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip"
unzip "${PKG_NAME}" -d /usr/local/
mv /usr/local/libtorch/ /usr/local/libtorch_cpu
ok "Successfully installed libtorch_cpu ${VERSION}"
 
##============================================================##
# libtorch_gpu
if [[ "${TARGET_ARCH}" == "x86_64" ]]; then
    VERSION="1.7.0-2"
    CHECKSUM="b64977ca4a13ab41599bac8a846e8782c67ded8d562fdf437f0e606cd5a3b588"
    PKG_NAME="libtorch_gpu-${VERSION}-cu111-linux-x86_64.tar.gz"
else # AArch64
    VERSION="1.6.0-1"
    PKG_NAME="libtorch_gpu-1.6.0-1-linux-aarch64.tar.gz"
    CHECKSUM="eeb5a223d9dbe40fe96f16e6711c49a3777cea2c0a8da2445d63e117fdad0385"
fi
 
# DOWNLOAD_LINK="https://apollo-system.cdn.bcebos.com/archive/6.0/${PKG_NAME}"
# download_if_not_cached "${PKG_NAME}" "${CHECKSUM}" "${DOWNLOAD_LINK}"
 
PKG_NAME="/apollo/libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip"
unzip "${PKG_NAME}" -d /usr/local/
mv /usr/local/libtorch/ /usr/local/libtorch_gpu
 
# Cleanup
rm -f "${PKG_NAME}"
ok "Successfully installed libtorch_gpu ${VERSION}"

可以看到首先把下载的命令注释掉了，然后自己写了一个 PKG_NAME，这个地方填的是我自己下载的 libtorch 的 zip 文件，然后解压到 / usr/local 目录下，它自己解压出来的是 libtorch 文件夹，后面我自己分别重命名为了 libtorch_cpu 和 libtorch_gpu 文件夹，然后直接

bash docker/build/installer/install_libtorch_new.sh

就会把原来的 / usr/local / 下面的 libtorch_cpu 和 libtorch_gpu 替换成我自己的版本，这里用的是 libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip 和 libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip。

PS：

再说一下 pytorch 的弱智指引，去他官网上下载的话实际上不能找到所有版本的 libtorch，只有这么个结果

你如果想要跟自己的 pytorch 版本匹配的话只能自己去下载，我用自己下载的举例。

CPU 版本：

https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-1.10.2+cpu.zip

这个命名方式很简单，直接把版本号替换就行。

GPU 版本：

https://download.pytorch.org/libtorch/cu111/libtorch-cxx11-abi-shared-with-deps-1.10.2+cu111.zip gpu 版本要先看自己的 cuda 版本，我的是 11.1，所以上面两处都是 cu111，然后在改 pytorch 版本号 1.10.2。这么操作就可以下载自己需要的 libtorch 版本了。