从头搭建Openstack运行环境（二）--虚机配置与虚拟网络设备搭建
3.双节点基础环境准备
3.1 虚机环境准备
3.1.1 在virtualbox环境安装centos7虚拟机两台
cpu、内存、磁盘和网络配置参考前面虚机配置信息
请选择"Minimal Install"最小化安装
配置eth0网卡ip
3.1.2 virtualbox宿主机配置
1）配置路由，保证两个网段虚机可以访问外网
sudo iptables -t nat -I POSTROUTING -s 10.255.255.0/24 -j MASQUERADE sudo iptables -t nat -I POSTROUTING -s 10.255.253.0/24 -j MASQUERADE sudo sysctl net.ipv4.ip_forward=1
3.1.3虚机安装后环境配置和依赖包安装
1）修改主机名
vi /etc/hostname
控制节点：ofs-network
计算节点：ofs-compute
2）配置grub启动项，设置网卡名为eth开头
vim /etc/sysconfig/grub 添加：
net.ifnames=0 biosdevname=0
grub2-mkconfig -o /boot/grub2/grub.cfg
3）关闭Network Manager管理
chkconfig NetworkManager off
4）配置ssh公私钥
ssh-keygen -t dsa -P '' -f /etc/ssh/ssh_host_dsa_key ssh-keygen -t rsa -P '' -f /etc/ssh/ssh_host_rsa_key
5）关闭selinux
vi /etc/selinux/config
6）重启虚机
reboot
7）安装openvswitch（网络节点、计算节点都需安装）
yum -y install wget openssl-devel kernel-devel yum -y groupinstall "Development Tools" adduser ovswitch su - ovswitch wget http://openvswitch.org/releases/openvswitch-2.4.0.tar.gz tar xfz openvswitch-2.4.0.tar.gz mkdir -p ~/rpmbuild/SOURCES cp openvswitch-2.4.0.tar.gz /home/ovswitch/rpmbuild/SOURCES/ rpmbuild -bb --without check ~/openvswitch-2.4.0/rhel/openvswitch.spec exit yum -y localinstall /home/ovswitch/rpmbuild/RPMS/x86_64/openvswitch-2.4.0-1.x86_64.rpm
8）安装qemu（只需计算节点安装）
yum -y install bridge-utils gcc zlib-devel glib2-devel pixman pixman-devel git clone git://git.qemu-project.org/qemu.git cd qemu ./configure --enable-debug --enable-vnc --enable-werror --target-list="x86_64-softmmu" make -j8 sudo make install
9）下载cirros镜像和kernel文件用于测试
wget http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img wget http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-kernel
4.网络节点部署
根据网络节点设计场景图，进行虚拟网络设备搭建
1）创建br-int集成网桥
ovs-vsctl --timeout=10 -- --may-exist add-br br-int ovs-vsctl --timeout=10 -- set-fail-mode br-int secure ovs-vsctl --timeout=10 -- --if-exists del-port br-int patch-tun
2）配置br-int流表
ovs-ofctl del-flows br-int ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=1,actions=normal" ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=0,table=22,actions=drop"
3）创建br-tun隧道网桥
ovs-vsctl --timeout=10 -- --if-exists del-br br-tun ovs-vsctl --timeout=10 -- --may-exist add-br br-tun ovs-vsctl --timeout=10 add-port br-int patch-tun -- set Interface patch-tun type=patch options:peer=patch-int ovs-vsctl --timeout=10 add-port br-tun patch-int -- set Interface patch-int type=patch options:peer=patch-tun
4）创建br-eth2 VLAN网桥
ovs-vsctl --timeout=10 -- --if-exists del-br br-eth2 ovs-vsctl --timeout=10 -- --may-exist add-br br-eth2 ovs-vsctl add-port br-eth2 eth2 ovs-vsctl --timeout=10 add-port br-int int-br-eth2 -- set Interface int-br-eth2 type=patch options:peer=phy-br-eth2 ovs-vsctl --timeout=10 add-port br-eth2 phy-br-eth2 -- set Interface phy-br-eth2 type=patch options:peer=int-br-eth2
4）配置br-int和br-eth2流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=2,in_port=2,actions=drop" ovs-ofctl del-flows br-eth2 ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=2,in_port=2,actions=drop" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=1,actions=normal"
5）配置br-tun流表
ovs-ofctl del-flows br-tun ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,in_port=1,actions=resubmit(,1)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00,actions=resubmit(,20)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=resubmit(,21)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=2,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=3,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=10,actions=learn(table=20,priority=1,hard_timeout=300,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=20,actions=resubmit(,21)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=21,actions=drop"
6）建立隧道连接
ovs-vsctl --timeout=10 -- --may-exist add-port br-tun gre-1 -- set Interface gre-1 type=gre options:remote_ip=10.255.254.101 options:local_ip=10.255.254.100 options:in_key=flow options:out_key=flow ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,in_port=2,actions=resubmit(,2)"
7）启动所有设备
ip link set dev br-int up ip link set dev br-tun up ip link set dev br-eth2 up ip link set dev eth2 up ip link set dev eth3 up
8)清空iptables
iptables -X iptables -F iptables -Z
5.计算节点部署
5.1网络架构创建
根据计算节点设计场景图，进行虚拟网络设备搭建
1）创建br-int网桥
ovs-vsctl --timeout=10 -- --if-exists del-port br-int patch-tun ovs-vsctl --timeout=10 -- --may-exist add-br br-int ovs-vsctl --timeout=10 -- set-fail-mode br-int secure
2）添加br-int网桥流表
ovs-ofctl del-flows br-int ovs-ofctl add-flow br-int hard_timeout=0,idle_timeout=0,priority=1,actions=normal ovs-ofctl add-flow br-int hard_timeout=0,idle_timeout=0,priority=0,table=22,actions=drop
3）创建br-tun网桥
ovs-vsctl --timeout=10 -- --if-exists del-br br-tun ovs-vsctl --timeout=10 -- --may-exist add-br br-tun ovs-vsctl --timeout=10 add-port br-int patch-tun -- set Interface patch-tun type=patch options:peer=patch-int ovs-vsctl --timeout=10 add-port br-tun patch-int -- set Interface patch-int type=patch options:peer=patch-tun
4）创建br-eth2网桥
ovs-vsctl --timeout=10 -- --if-exists del-br br-eth2 ovs-vsctl --timeout=10 -- --may-exist add-br br-eth2 ovs-vsctl add-port br-eth2 eth2 ovs-vsctl --timeout=10 add-port br-int int-br-eth2 -- set Interface int-br-eth2 type=patch options:peer=phy-br-eth2 ovs-vsctl --timeout=10 add-port br-eth2 phy-br-eth2 -- set Interface phy-br-eth2 type=patch options:peer=int-br-eth2
5）为br-int和br-eth2网桥添加流表
ovs-ofctl add-flow br-int "hard_timeout=0,idle_timeout=0,priority=2,in_port=2,actions=drop" ovs-ofctl del-flows br-eth2 ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=2,in_port=2,actions=drop" ovs-ofctl add-flow br-eth2 "hard_timeout=0,idle_timeout=0,priority=1,actions=normal"
6）为br-tun网桥添加流表
ovs-ofctl del-flows br-tun ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,in_port=1,actions=resubmit(,1)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=1,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00,actions=resubmit(,20)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=resubmit(,21)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=2,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=3,actions=drop" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,table=10,actions=learn(table=20,priority=1,hard_timeout=300,NXM_OF_VLAN_TCI[0..11],NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:0->NXM_OF_VLAN_TCI[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=20,actions=resubmit(,21)" ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=0,table=21,actions=drop"
7）建立隧道连接
ovs-vsctl --timeout=10 -- --may-exist add-port br-tun gre-1 -- set Interface gre-1 type=gre options:remote_ip=10.255.254.100 options:local_ip=10.255.254.101 options:in_key=flow options:out_key=flow ovs-ofctl add-flow br-tun "hard_timeout=0,idle_timeout=0,priority=1,in_port=2,actions=resubmit(,2)"
8）启动设备
ip link set dev br-int up ip link set dev br-tun up ip link set dev br-eth2 up ip link set dev eth2 up
9）清空iptables
iptables -X iptables -F iptables -Z
以上操作完成了对网络节点和计算节点虚拟网络设备框架的搭建，下期将介绍在此双节点环境上部署虚机相关操作内容，敬请期待。
参考资料：
SammyLiu的《Neutron 理解》系列 http://www.cnblogs.com/sammyliu/p/4622563.html
深入理解Neutron -- OpenStack 网络实现 https://www.gitbook.com/book/yeasy/openstack_understand_neutron/details
作者简介：赵俊峰，现为北京新云东方系统科技有限责任公司 云计算部Openstack开发工程师。主要从事Power和x86混合环境下Openstack相关计算、网络、存储相关服务软件开发和系统架构设计工作。
