2016-08-26 ib1

ib0不见了
 
0000:04:00.0 'MT27500 Family [ConnectX-3]'
 
做法：如果直接绑定不行，那就试试vf的那个
总之试试在先
 
connectx_port_config修改ib网卡的模式为Ethernet
然后
 
2016-08-28 setup.sh igb_uio+vfio
只有vfio不行
 
librte_pmd_mlx4: cannot access device, is mlx4_ib loaded?
 
libibverbs 
If you intend to use dynamic libraries, you should uninstall libibverbs, libmlx4 and related packages first.
 
 
 
0828新试：
Ethernet+mtcp看看行不行
 
还是不行，应该最终还是不行。下周试试dpdk2.2
 
 2016-08-29 some commands
mstflint -d 04:00.0 q
 
ConnectX PCI devices :
|----------------------------|
| 1             0000:04:00.0 |
|----------------------------|
 
Before port change:
auto (ib)
 
死活只有一个端口，没有俩端口~
 
./configure --with-mlx4_fc-mod  --with-mlx4_inf-mod --with-core-mod --with-user_mad-mod --with-user_access-mod --with-addr_trans-mod --with-mlx4-mod --with-mlx4_en-mod --with-ipoib-mod --with-srp-mod --with-rds-mod --with-iser-mod --kernel-version=3.10.0-mlx --modules-dir=/lib/modules/3.10.0-mlx/ --kernel-sources=/work/kernel-3.10.0-123.el7.src/linux-3.10.0-123.el7
 
/usr/bin/ld: cannot find /root/pool/install/usr/local/lib: File format not recognized
collect2: error: ld returned 1 exit status
/root/dpdk/mk/rte.app.mk:230: recipe for target 'test' failed
make[4]: *** [test] Error 1
/root/dpdk/mk/rte.subdir.mk:61: recipe for target 'test' failed
make[3]: *** [test] Error 2
/root/dpdk/mk/rte.sdkbuild.mk:78: recipe for target 'app' failed
make[2]: *** [app] Error 2
/root/dpdk/mk/rte.sdkroot.mk:123: recipe for target 'all' failed
make[1]: *** [all] Error 2
Makefile:12: recipe for target 'all' failed
make: *** [all] Error 2
 

2016-08-29 ib0

-w 0000:04:00.0
 
1. Install OFED, run /etc/init.d/openibd restart
2. Compile DPDK with the CONFIG_RTE_LIBRTE_MLX4_PMD=y   <dpdkdir>/config/common_base
3. Set up hugepages
4. Load kernel modules
5. Run testpmd
 
modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
 
 
export RTE_SDK=/root/dpdk
export RTE_TARGET=x86_64-native-linuxapp-gcc
make
 
mkdir -p /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
/etc/init.d/openibd restart
ls -d /sys/class/net/*/device/infiniband_verbs/uverbs* | cut -d / -f 5
 
(cd "/sys/class/net/eth2/device/" && pwd -P) | sed -n 's,.*/\(.*\),-w \1,p'
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/nr_hugepages
./tools/dpdk-devbind.py --bind=mlx4_core 04:00.0
 
Cannot open /sys/bus/pci/drivers/mlnx_core/bind
 
两个地方的.config文件都改了，希望可以
 
2016-08-29 useless
librte_pmd_mlx4: cannot access device, is mlx4_ib loaded?
 
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1003 librte_pmd_mlx4
PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_0" (VF: false)
PMD: librte_pmd_mlx4: 1 port(s) detected
PMD: librte_pmd_mlx4: port 1 MAC address is 46:c9:20:00:36:f0
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=203456, size=2176, socket=0
 
Warning! Cannot handle an odd number of ports with the current port topology. Configuration must be changed to have an even number of ports, or relaunch application with --port-topology=chained
 
Configuring Port 0 (socket 0)
PMD: librte_pmd_mlx4: 0xae7840: TX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: 0xae7840: RX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: mempool 0x7f454147dec0: not virtually contiguous
PMD: librte_pmd_mlx4: mempool 0x7f454147dec0: not virtually contiguous
PMD: librte_pmd_mlx4: 0xae7840: MR creation failure: Invalid argument
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed
 
 
root@node112~/dpdk# ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff00 -n 4 -w 0000:04:00.0 -w 0000:03:00.1 -- -i
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.1 on NUMA socket -1
EAL:   probe driver: 8086:1510 rte_igb_pmd
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1003 librte_pmd_mlx4
PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_0" (VF: false)
PMD: librte_pmd_mlx4: 1 port(s) detected
PMD: librte_pmd_mlx4: port 1 MAC address is 46:c9:20:00:36:f0
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=203456, size=2176, socket=0
 
Warning! Cannot handle an odd number of ports with the current port topology. Configuration must be changed to have an even number of ports, or relaunch application with --port-topology=chained
 
Configuring Port 0 (socket 0)
PMD: librte_pmd_mlx4: 0xae7840: TX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: 0xae7840: RX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: mempool 0x7fb11447dec0: not virtually contiguous
PMD: librte_pmd_mlx4: mempool 0x7fb11447dec0: not virtually contiguous
PMD: librte_pmd_mlx4: 0xae7840: MR creation failure: Invalid argument
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed
 
root@node112~/dpdk# ./examples/l2fwd/build/l2fwd -c1 -n1 -- -p1
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   probe driver: 8086:1510 rte_igb_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket -1
EAL:   probe driver: 8086:1510 rte_igb_pmd
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL:   probe driver: 15b3:1003 librte_pmd_mlx4
PMD: librte_pmd_mlx4: PCI information matches, using device "mlx4_0" (VF: false)
PMD: librte_pmd_mlx4: 1 port(s) detected
PMD: librte_pmd_mlx4: port 1 MAC address is 46:c9:20:00:36:f0
Notice: odd number of ports in portmask.
Lcore 0: RX port 0
Initializing port 0... PMD: librte_pmd_mlx4: 0x84a9c0: TX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: 0x84a9c0: RX queues number update: 0 -> 1
PMD: librte_pmd_mlx4: 0x7fff41b6c8e0: flow configuration failed, errno=38: Function not implemented
PMD: librte_pmd_mlx4: 0x84a9c0: QP flow attachment failed: Function not implemented
EAL: Error - exiting with code: 1
  Cause: rte_eth_rx_queue_setup:err=-38, port=0
 
2016-08-30 details
整理一：
1, <dpdk>/config/common_base: set CONFIG_RTE_LIBRTE_MLX4_PMD = y
2, modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
3, /etc/init.d/openibd restart
4, build dpdk & setup hugepages
 
不需要将网卡用igb_uio或vfio绑定，直接运行dpdk的l2fwd程序能够自动找到对应网卡
 
整理二：
 
 
./build/build/app/test-pmd/testpmd -c 0xf -n 4 -w 0000:01:00.0 -- --rxq=2 --txq=2 -i
 
2016-12-02 next
要不要试一试ofed2.4看看，虽然它说不能在debian8.3上面搞，但是可以试试，它的内核可以支持到3.18，目前debian的内核是3.16
 
Failed to check for authentication: GDBus.Error:org.freedesktop.PolicyKit1.Error.
Failed: Action org.freedesktop.packagekit.system-update is not registered
 
2016-12-19 pica8
ovs-vsctl add-port br0 ge-1/1/1 vlan_mode=access tag=1 -- set Interface
ge-1/1/1 type=pica8
 
ovs-vsctl add-port br0 te-1/1/49 vlan_mode=trunk tag=1 -- set Interface
te-1/1/49 type=pica8 options:is_dac=true
 
ovs-ofctl show br0 #显示所有端口信息
 
ryu-manager --ofp-tcp-listen-port 6633 ryu.app.simple_switch_13
 
 
leave minicom ctrl+a，x