Xinference 基于 Docker 部署

1、Docker、Docker-Compose安装

https://www.cnblogs.com/a120608yby/p/9883175.html
https://www.cnblogs.com/a120608yby/p/14582853.html

2、安装NVIDIA Container Toolkit

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

3、基于Docker-Compose部署Xinference

# vim /data/docker/docker-xinference/docker-compose.yml
services:
  xinference:
    image: xprobe/xinference:latest
    container_name: xinference
    ports:
      - "9997:9997"
    restart: unless-stopped
    networks:
      - ops_default
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              count: all
    environment:
      - XINFERENCE_HOME=/data
    volumes:
      - ./data:/data
    command: xinference-local -H 0.0.0.0

networks:
  ops_default:
    external: false

4、启动服务

docker compose up -d

5、访问

访问UI： http://部署主机IP:9997/ui
访问API文档： http://部署主机IP:9997/docs

参考：

https://inference.readthedocs.io/zh-cn/latest/getting_started/