Proxmox VE 9.0直通显卡配置 (一)

整体文章
Proxmox VE 9.0直通显卡配置 (一)
Proxmox VE 9.0直通显卡配置 (二)

1.机器信息

• 显卡信息为RX570 8G版本(下面显示有问题)

root@pve:~# neowofetch
         .://:`              `://:.            root@pve
       `hMMMMMMd/          /dMMMMMMh`          --------
        `sMMMMMMMd:      :mMMMMMMMs`           OS: Proxmox VE 9.0.5 x86_64
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`   Kernel: 6.14.8-2-pve
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`   Uptime: 10 mins
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`     Packages: 823 (dpkg)
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.       Shell: bash 5.2.37
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         Terminal: /dev/pts/0
        -+ooooooo/.`sMMs`./ooooooo+-           CPU: AMD Ryzen 7 5700X (16) @ 4.6GHz
          :oooooooo/`..`/oooooooo:             GPU: AMD ATI Radeon RX 570 Pulse 4GB
          :oooooooo/`..`/oooooooo:             Memory: 3.44 GiB / 125.71 GiB (2%)
        -+ooooooo/.`sMMs`./ooooooo+-           Network: 1 Gbps
      .+ooooooo+-`oNMMMMNo`-+ooooooo+.         BIOS: American Megatrends Inc. 5.17 (03/01/2023)
    ./ooooooo+- +NMMMMMMMMN+ -+ooooooo/.
  `/oooooooo:`:mMMMMMMMMMMMMm:`:oooooooo/`
`:oooooooo/`-hMMMMMMMyyMMMMMMMh-`/oooooooo:`
`-/+oo+/:`.yMMMMMMMh-  -hMMMMMMMy.`:/+oo+/-`
        `sMMMMMMMm:      :dMMMMMMMs`
       `hMMMMMMd/          /dMMMMMMh`
         `://:`              `://:`

root@pve:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 13 (trixie)
Release:        13
Codename:       trixie
root@pve:~#

2.物理机开启IOMMU等功能

3.PVE开启显卡直通

3.1. 配置grub

• 在GRUB_CMDLINE_LINUX_DEFAULT行中添加video=efifb:off内容

root@pve:~# cat /etc/default/grub | grep GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet video=efifb:off"
root@pve:~#
root@pve:~# update-grub
root@pve:~# reboot

3.2.配置vfio

• 在/etc/modules中设置加载vfio、vfio_iommu_type1、vfio_pci模块

root@pve:~# cat /etc/modules
# /etc/modules is obsolete and has been replaced by /etc/modules-load.d/.
# Please see modules-load.d(5) and modprobe.d(5) for details.
#
# Updating this file still works, but it is undocumented and unsupported.

# add vfio
vfio
vfio_iommu_type1
vfio_pci
root@pve:~#

• 配置VFIO-PCI 设备绑定规则

# 获取设备PCI的值，可通过 pvesh get /nodes/pve/hardware/pci --pci-class-blacklist ""命令获取
root@pve:~# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:67df,1002:aaf0
root@pve:~#

3.3.黑名单规则,禁止PVE主机加载特定内核模块

root@pve:~# cat /etc/modprobe.d/blacklist.conf
blacklist amdgpu
blacklist radeon
blacklist nouveau
blacklist amdkfd
root@pve:~#

3.4. 更新内核

root@pve:~# update-initramfs -u -k all
root@pve:~# reboot

3.5.完成vfio直通配置验证

root@pve:~# lspci -nnk -s 0b:00.0
0b:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
        Subsystem: Sapphire Technology Limited Radeon RX 570 Pulse 4GB [1da2:e353]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
root@pve:~# lspci -nnk -s 0b:00.1
0b:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
        Subsystem: Sapphire Technology Limited Device [1da2:aaf0]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
root@pve:~#
root@pve:~# dmesg | grep -e DMAR -e IOMMU
[    0.892490] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.896135] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
root@pve:~# dmesg | grep IOMMU
[    0.892490] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.896135] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
root@pve:~#