centos7.6安装dpdk-stable-19.11.10
1. 安装dpdk-stable-19.11.10需要操作系统内核版本大于3.2，如果版本过低，可以通过以下方式升级
1 #cat /etc/redhat-release 2 CentOS Linux release 7.6.1810 (Core) 4 #uname -r 查看一下系统版本 5 3.10.0-957.el7.x86_64 6 #rpm -qa kernel 通过rpm命令查看我所安装的内核版本 7 kernel-3.10.0-957.el7.x86_64 8 #ls /usr/src/kernels/ 查看有没有相应的内核开发包 9 空 如果该目录下没有系统内核源码，执行以下操作 10 #yum install kernel-devel 安装内核头文件后 11 #ls /usr/src/kernels/ 12 3.10.0-1062.12.1.el7.x86_64 13 两个版本号不一致，进行升级并重启 18 #yum -y update kernel kernel-devel 19 #reboot 20 再此查看，版本号一致，问题解决 21 #ls /usr/src/kernels/ 22 3.10.0-1062.12.1.el7.x86_64 23 #uname -r 24 10.0-1062.12.1.el7.x86_64
2. 安装依赖包
sudo yum install gcc make yum install libpcap libpcap-devel yum install numactl numactl-devel
3. 从dpdk官网下载dpdk压缩包并解压
将dpdk压缩包下载到/home目录下并解压
wget http://fast.dpdk.org/rel/dpdk-19.11.10.tar.xz
xz -d dpdk-19.11.10.tar.xz
tar -xvf dpdk-19.11.10.tar
4. 编译
1. 在解压的dpdk-19.11.10目录下，创建dpdkenv文件 在文件中写入： export RTE_SDK='/home/dpdk-19.11.10' export RTE_TARGET=x86_64-native-linuxapp-gcc #(对于64位机用这个命令，对于32位机用i686-native-linuxapp-gcc) 2. 保存后，在命令行执行： source dpdkenv 3. 查看环境变量是否设置好： env |grep RTE 4. 关闭要绑定的网卡，否则绑定dpdk时不成功 ifconfig ens37 down 5. 进入到dpdk-19.11.10/usertools目录下 cd dpdk-19.11.10/usertools 执行./dpdk-setup.sh 会输出一列可选操作： Step 1: Select the DPDK environment to build ---------------------------------------------------------- [1] arm64-armada-linuxapp-gcc [2] arm64-armada-linux-gcc [3] arm64-armv8a-linuxapp-clang [4] arm64-armv8a-linuxapp-gcc [5] arm64-armv8a-linux-clang [6] arm64-armv8a-linux-gcc [7] arm64-bluefield-linuxapp-gcc [8] arm64-bluefield-linux-gcc [9] arm64-dpaa-linuxapp-gcc [10] arm64-dpaa-linux-gcc [11] arm64-emag-linuxapp-gcc [12] arm64-emag-linux-gcc [13] arm64-graviton2-linuxapp-gcc [14] arm64-graviton2-linux-gcc [15] arm64-n1sdp-linuxapp-gcc [16] arm64-n1sdp-linux-gcc [17] arm64-octeontx2-linuxapp-gcc [18] arm64-octeontx2-linux-gcc [19] arm64-stingray-linuxapp-gcc [20] arm64-stingray-linux-gcc [21] arm64-thunderx2-linuxapp-gcc [22] arm64-thunderx2-linux-gcc [23] arm64-thunderx-linuxapp-gcc [24] arm64-thunderx-linux-gcc [25] arm64-xgene1-linuxapp-gcc [26] arm64-xgene1-linux-gcc [27] arm-armv7a-linuxapp-gcc [28] arm-armv7a-linux-gcc [29] graviton2 [30] i686-native-linuxapp-gcc [31] i686-native-linuxapp-icc [32] i686-native-linux-gcc [33] i686-native-linux-icc [34] ppc_64-power8-linuxapp-gcc [35] ppc_64-power8-linux-gcc [36] x86_64-native-bsdapp-clang [37] x86_64-native-bsdapp-gcc [38] x86_64-native-freebsd-clang [39] x86_64-native-freebsd-gcc [40] x86_64-native-linuxapp-clang [41] x86_64-native-linuxapp-gcc [42] x86_64-native-linuxapp-icc [43] x86_64-native-linux-clang [44] x86_64-native-linux-gcc [45] x86_64-native-linux-icc [46] x86_x32-native-linuxapp-gcc [47] x86_x32-native-linux-gcc ---------------------------------------------------------- Step 2: Setup linux environment ---------------------------------------------------------- [48] Insert IGB UIO module [49] Insert VFIO module [50] Insert KNI module [51] Setup hugepage mappings for non-NUMA systems [52] Setup hugepage mappings for NUMA systems [53] Display current Ethernet/Baseband/Crypto device settings [54] Bind Ethernet/Baseband/Crypto device to IGB UIO module [55] Bind Ethernet/Baseband/Crypto device to VFIO module [56] Setup VFIO permissions ---------------------------------------------------------- Step 3: Run test application for linux environment ---------------------------------------------------------- [57] Run test application ($RTE_TARGET/app/test) [58] Run testpmd application in interactive mode ($RTE_TARGET/app/testpmd) ---------------------------------------------------------- Step 4: Other tools ---------------------------------------------------------- [59] List hugepage info from /proc/meminfo ---------------------------------------------------------- Step 5: Uninstall and system cleanup ---------------------------------------------------------- [60] Unbind devices from IGB UIO or VFIO driver [61] Remove IGB UIO module [62] Remove VFIO module [63] Remove KNI module [64] Remove hugepage mappings [65] Exit Script Option: 在最下面的 Option: 处输入 41 会选择适合x86_64机器的gcc编译器，如果是其他架构的机器，需要选择对应的其他编译器 编译中，稍等片刻...... 编译完成后会出现 Build complete [x86_64-native-linuxapp-gcc] Installation cannot run with T defined and DESTDIR undefined ------------------------------------------------------------------------------ RTE_TARGET exported as x86_64-native-linuxapp-gcc ------------------------------------------------------------------------------ Press enter to continue ... 按下回车键，继续选择其他操作： Option: 48 加载igb uio模块 Unloading any existing DPDK UIO module Loading DPDK UIO module Press enter to continue ... 按下回车键，继续选择其他操作： Option: 52 配置大页 Removing currently reserved hugepages Unmounting /mnt/huge and removing directory Input the number of 2048kB hugepages for each node Example: to have 128MB of hugepages available per node in a 2MB huge page system, enter '64' to reserve 64 * 2MB pages on each node Number of pages for node0: （在此输入2048） 2048 Reserving hugepages Creating /mnt/huge and mounting as hugetlbfs Press enter to continue ... 按下回车键，继续选择其他操作： Option: 53 列出所有网卡 Network devices using kernel driver =================================== 0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio *Active* 0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens38 drv=e1000 unused=igb_uio *Active* 0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens39 drv=e1000 unused=igb_uio *Active* No 'Baseband' devices detected ============================== No 'Crypto' devices detected ============================ No 'Eventdev' devices detected ============================== No 'Mempool' devices detected ============================= No 'Compress' devices detected ============================== No 'Misc (rawdev)' devices detected =================================== Press enter to continue ... 按下回车键，继续选择其他操作： Option: 54 绑定网卡 Network devices using kernel driver =================================== 0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio *Active* 0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens38 drv=e1000 unused=igb_uio *Active* 0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens39 drv=e1000 unused=igb_uio *Active* No 'Baseband' devices detected ============================== No 'Crypto' devices detected ============================ No 'Eventdev' devices detected ============================== No 'Mempool' devices detected ============================= No 'Compress' devices detected ============================== No 'Misc (rawdev)' devices detected =================================== Enter PCI address of device to bind to IGB UIO driver: （在此输入网卡pci号，就是上边列出来的网卡信息，把开头的0000:去掉）02:06.0 成功后会输出： ok Press enter to continue ... 按下回车键，继续选择其他操作： Option: 53 查看网卡绑定情况 Network devices using DPDK-compatible driver ============================================ 出现这一行表示绑定成功 0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=e1000 Network devices using kernel driver =================================== 0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio *Active* 0000:02:07.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens39 drv=e1000 unused=igb_uio *Active* No 'Baseband' devices detected ============================== No 'Crypto' devices detected ============================ No 'Eventdev' devices detected ============================== No 'Mempool' devices detected ============================= No 'Compress' devices detected ============================== No 'Misc (rawdev)' devices detected =================================== Press enter to continue ... 按下回车键，继续选择其他操作： Option: 57 进行简单测试： Enter hex bitmask of cores to execute test app on Example: to execute app on cores 0 to 7, enter 0xff bitmask: 0xff （在此输入0xff ） Launching app EAL: Detected 8 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:07.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em APP: HPET is not enabled, using TSC as default timer 输出以上内容表示安装dpdk成功 RTE>>quit（在此输入quit） Press enter to continue .... 按下回车键，继续选择其他操作： Option: 65 退出安装程序，至此，dpdk安装成功。
5. helloword测试
进入到dpdk-stable-19.11.10/examples/helloworld目录 执行： make 编译完成后，进入当前目录下的build目录，执行： ./helloworld 输出： EAL: Detected 8 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No available hugepages reported in hugepages-1048576kB # 这个是正常的告警，不影响使用的，详见https://stackoverflow.com/questions/64698701/what-does-eal-no-available-hugepages-reported-in-hugepages-1048576kb-mean-in EAL: Probing VFIO support... EAL: PCI device 0000:02:01.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:05.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:06.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em EAL: PCI device 0000:02:07.0 on NUMA socket -1 EAL: probe driver: 8086:100f net_e1000_em
（有可能在这里出现：EAL: Error reading from file descriptor 43: Input/output error）
是由于dpdk不支持该网卡导致，需要修改一行代码，跳过dpdk pci 检查。
vim dpdk-stable-19.11.10/kernel/linux/igb_uio/igb_uio.c
将(pci_intx_mask_supported(udev->pdev))修改为 (pci_intx_mask_supported(udev->pdev)||true)
再重新走一遍上边的编译流程即可。
hello from core 1 hello from core 2 hello from core 3 hello from core 4 hello from core 5 hello from core 6 hello from core 7 hello from core 0 表示成功
部分报错，参考文档 https://blog.csdn.net/qq_40443457/article/details/104780480
