ubuntu docker运行大模型

安装docker

安装依赖
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg lsb-release

添加官方 Docker 源

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg |
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo
"deb [arch=$(dpkg --print-architecture)
signed-by=/etc/apt/keyrings/docker.gpg]
https://download.docker.com/linux/ubuntu
$(lsb_release -cs) stable" |
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

安装 Docker Engine

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

启动测试

sudo systemctl enable docker
sudo systemctl start docker
sudo docker run hello-world

配置NVIDIA Container Toolkit

安装 NVIDIA Container Toolkit（让 Docker 支持 GPU）
浏览器打开地址后下载GPG密钥 https://nvidia.github.io/libnvidia-container/gpgkey

登陆github:https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.18.0
下载toolkit包
tar -xzvf nvidia-container-toolkit_1.18.0_deb_amd64.tar.gz
sudo dpkg -i ./*.deb

nvidia-ctk --version

配置 Docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

测试 GPU 容器运行

sudo docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi

docker info | grep -i runtime

应看到: Runtimes: nvidia runc

cat /etc/docker/daemon.json
{
"registry-mirrors": [
"https://docker.m.daocloud.io",
"https://mirror.baidubce.com",
"https://ccr.ccs.tencentyun.com",
"https://hub-mirror.c.163.com",
"https://registry.cn-hangzhou.aliyuncs.com"
],
"runtimes": {
"nvidia": {
"path": "nvidia-container-runtime",
"args": []
}
}
}

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html