Open vSwitch构建Docker跨主机网络

环境说明:

主机名操作系统宿主机IPDocker IP 
ovs01 ubuntu 18.04 192.168.168.10 172.17.0.1
ovs02 ubuntu 18.04  192.168.168.11 172.17.1.2

 

 

 

 

安装配置OVS网络:

1、安装docker-ce (安装过程此处略)

2、设置docker0网段(ovs02同样操作,bip不同)

$ sudo vi /etc/docker/daemon.json
{
    "bip":"172.17.0.1/24"
}

$ sudo systemctl restart docker

3、安装openvswitch-switch和bridge-utils

$ sudo apt-get -y install openvswitch-switch bridge-utils

4、查看ovs运行状态

$ sudo ps -ea | grep ovs
  1526 ?        00:00:00 ovsdb-server
  1593 ?        00:00:00 ovs-vswitchd

5、查看ovs版本信息和ovs支持的OpenFlow协议的版本

$ sudo ovs-appctl --version
ovs-appctl (Open vSwitch) 2.9.5

$ sudo ovs-ofctl --version
ovs-ofctl (Open vSwitch) 2.9.5
OpenFlow versions 0x1:0x5

6、创建br0网桥并激活

$ sudo ovs-vsctl add-br br0
$ sudo ip link set dev br0 up

7、创建gre隧道(remote_ip为peer宿主机ip)

$ sudo ovs-vsctl add-port br0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.168.11     //ovs01配置

$ sudo ovs-vsctl add-port br0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.168.10     //ovs02配置

注:如有多台docker主机需要构建网络创建多个gre隧道

8、将br0作为接口加入docker0网桥

$ sudo brctl addif docker0 br0
$ sudo brctl stp docker0 on

9、查看网桥配置

$ sudo ovs-vsctl show
cedc63c1-97d6-4e5e-bdf0-3efc0a5b7aa4
    Bridge "br0"
        Port "br0"
            Interface "br0"
                type: internal
        Port "vxlan0"
            Interface "gre0"
                type: gre
                options: {remote_ip="192.168.168.11"}
    ovs_version: "2.9.5"

$ brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.02425f251c20       no              br0

10、添加静态路由(网段地址为peer Docker网段)

$ sudo ip route add 172.17.1.0/24 dev docker0  //ovs01添加peer docker net
$ sudo ip route add 172.17.0.0/24 dev docker0  //ovs02添加peer docker net

11、测试连通性

$ docker run -it busybox:1.28.3 /bin/sh       //ovs01测试
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
4: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: eth0@if11: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.2/24 brd 172.17.0.255 scope global eth0
       valid_lft forever preferred_lft forever
/ # ping 172.17.1.2
PING 172.17.1.2 (172.17.1.2): 56 data bytes
64 bytes from 172.17.1.2: seq=0 ttl=63 time=3.302 ms
64 bytes from 172.17.1.2: seq=1 ttl=63 time=0.824 ms

$ docker run -it busybox:1.28.3 /bin/sh      //ovs02测试
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: gre0@NONE: <NOARP> mtu 1476 qdisc noop qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
3: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
4: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1450 qdisc noop qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: eth0@if11: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 02:42:ac:11:01:02 brd ff:ff:ff:ff:ff:ff
    inet 172.17.1.2/24 brd 172.17.1.255 scope global eth0
       valid_lft forever preferred_lft forever
/ # ping 172.17.0.2
PING 172.17.0.2 (172.17.0.2): 56 data bytes
64 bytes from 172.17.0.2: seq=0 ttl=63 time=1.903 ms
64 bytes from 172.17.0.2: seq=1 ttl=63 time=0.765 ms

12、网桥配置和添加路由配置重启宿主机后会失效,写成shell脚本,重启后执行

$ sudo cat > add_bridge.sh <<EOF   //ovs01配置
#!/bin/bash
sudo ip link set dev br0 up
sudo brctl addif docker0 br0
sudo ip route add 172.17.1.0/24 dev docker0
EOF
$ sudo chmod +x add_bridge.sh

$ sudo cat > add_bridge.sh <<EOF   //ovs02配置
#!/bin/bash
sudo ip link set dev br0 up
sudo brctl addif docker0 br0
sudo ip route add 172.17.0.0/24 dev docker0
EOF
$ sudo chmod +x add_bridge.sh

CentOS7 OVS安装并生成RPM安装包

1、安装依赖包

yum -y install make gcc openssl-devel autoconf automake rpm-build redhat-rpm-config python-devel kernel-devel kernel-debug-devel libtool bridge-utils

2、下载OVS二进制安装包

# mkdir -p ~/rpmbuild/SOURCES
# wget https://www.openvswitch.org/releases/openvswitch-2.5.9.tar.gz -P ~/rpmbuild/SOURCES

3、生成OVS RPM安装包

# cd ~/rpmbuild/SOURCES
# tar -xvf openvswitch-2.5.9.tar.gz
# sed 's/openvswitch-kmod, //g' openvswitch-2.5.9/rhel/openvswitch.spec > openvswitch-2.5.9/rhel/openvswitch_no_kmod.spec

# rpmbuild -bb --nocheck openvswitch-2.12.0/rhel/openvswitch_no_kmod.spec

4、安装OVS

# yum localinstall ~/rpmbuild/RPMS/x86_64/openvswitch-2.5.9-1.x86_64.rpm

下载备份OVS RPM包,可在其它CentOS系统直接使用

5、启动OVS服务

# service openvswitch start                                             
Starting openvswitch (via systemctl):                      [  OK  ]

# service openvswitch status
ovsdb-server is running with pid 7004
ovs-vswitchd is running with pid 7024

# chkconfig --add openvswitch
# chkconfig openvswitch on

# tail -50f /var/log/messages
Jan 19 11:07:39 ovs yum[6922]: Installed: openvswitch-2.5.9-1.x86_64
Jan 19 11:07:52 ovs systemd: Starting LSB: Open vSwitch switch...
Jan 19 11:07:52 ovs openvswitch: /etc/openvswitch/conf.db does not exist ... (warning).
Jan 19 11:07:52 ovs openvswitch: Creating empty database /etc/openvswitch/conf.db [  OK  ]
Jan 19 11:07:52 ovs openvswitch: Starting ovsdb-server [  OK  ]
Jan 19 11:07:52 ovs ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait -- init -- set Open_vSwitch . db-version=7.12.1
Jan 19 11:07:52 ovs ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl --no-wait set Open_vSwitch . ovs-version=2.5.9 "external-ids:system-id=\"5aed6a14-bad2-438b-b012-c3dcbcb817fc\"" "system-type=\"unknown\"" "system-version=\"unknown\""
Jan 19 11:07:52 ovs openvswitch: Configuring Open vSwitch system IDs [  OK  ]
Jan 19 11:07:52 ovs kernel: nf_conntrack version 0.5.0 (7928 buckets, 31712 max)
Jan 19 11:07:52 ovs kernel: openvswitch: Open vSwitch switching datapath
Jan 19 11:07:52 ovs openvswitch: Inserting openvswitch module [  OK  ]
Jan 19 11:07:52 ovs openvswitch: Starting ovs-vswitchd [  OK  ]
Jan 19 11:07:52 ovs openvswitch: Enabling remote OVSDB managers [  OK  ]
Jan 19 11:07:52 ovs systemd: Started LSB: Open vSwitch switch.

 

posted @ 2020-01-18 21:56  不倒翁Jason  阅读(348)  评论(0编辑  收藏  举报