Openstack Neutron 网络配置(Linuxbridge)

目录

 Neutron官方配置文档 https://wiki.openstack.org/wiki/Neutron

Note

在修改了 ml2_conf.ini, linuxbridge_agent.ini, openvswitch_agent.ini 配置文件后都需要重启Neutron相关服务

 

Neutron CorePlugins

Core Plugins 主要是ml2 plugins, ml2 plugins又分为type drivers和mechanism drivers:

type driver又分为： local,float,vlan,vxlan,gre,geneve

mechanism drivers又分为： l2Population(在overlay网络中用到), LinuxBridge, OpenvSwitch ...

L2 population is mechanism driver for ML2 plugin which tends to leverage the 
implementation of overlay networks. By populating the forwarding tables of virtual 
switches (LinuxBridge or OVS), l2population mech driver will decrease broadcast traffics 
inside the physical networks fabric while using overlays networks (VXLan, GRE). Full 
specifications of the blueprint are available here.

 LinuxBridge和OpenvSwitch这两个Driver被定义在：

/usr/lib/python2.7/dist-packages/neutron/agent/linux/interface.py中的两个类

BrdigeInterfaceDriver 和 OVSInterfaceDriver

 

物理网卡配置

需要注意的是，在配置所有neutron网络以前，需要将需要使用到的物理网络初始化，示例如下：

auto lo
iface lo inet loopback

auto eth0 //管理网卡
iface eth0 inet static
address 192.168.20.180
netmask 255.255.255.0
gateway 192.168.20.1

auto eth1 // vlan/vlxan/gre网卡
iface eth1 inet manual

auto eth2 //flat,external网卡
iface eth2 inet manual

 

flat network配置

flat network 是不带tag的网络，要求宿主机的物理网卡直接与Linux Bridge连接，这意味着每个flat network都会独占一个物理网卡，flat network通常用来作为external network。

flat network涉及两个配置文件：

1、/etc/neutron/plugins/ml2/ml2_conf.ini(在neutron server上修改)

[ml2_type_flat]
...
flat_networks = external

2、/etc/neutron/plugins/ml2/linuxbridge_agent.ini(在所有compute节点修改)

[linux_bridge]
...
physical_interface_mappings = external:eth2

3、在创建flat网络的时候需要注意填写

4、创建完成后，使用brctl show进行查看

root@server01:~# brctl show
bridge name	bridge id		STP enabled	interfaces
brq790d6129-44		8000.005056881d5d	no		eth2
							tapaeb0993e-bf
virbr0		8000.5254000b949d	yes		virbr0-nic

 

vlan network配置

vlan network涉及两个配置文件：

1、/etc/neutron/plugins/ml2/ml2_conf.ini(在neutron server上修改)

[ml2]
...
type_drivers=local,flat,vlan,vxlan
tenant_network_types=vlan
mechanism_drivers=linuxbridge 
extension_drivers=port_security

[ml2_type_vlan]
...
network_vlan_ranges=default:100:200

2、/etc/neutron/plugins/ml2/linuxbridge_agent.ini(在所有compute节点上修改)

[linux_bridge]
...
physical_interface_mappings=default:eth1

3、在Horizon创建vlan后查看结果

root@server01:~# brctl show
bridge name	bridge id		STP enabled	interfaces
brq2c2db99a-a8		8000.005056887e32	no		eth1.101
							tap02e9aeef-21
							tap63b1b082-85
							tapb5e7b0d9-05

 

DHCP配置

配置文件/etc/neutron/dhcp_agent.ini

//LinuxBrdige
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

//OpenvSwitch
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq

Neutron的dhcp是 dnsmasq来提供的，dnsmasq是一个提供DHCP和DNS服务的开源软件。

Neutron通过namespance为每个network提供独立的DHCP服务和路由服务，从而允许租户创建重叠的网络。

在Neutron创建DHCP后，其配置信息存放路径：/var/lib/neutron/dhcp/

 

使用neutron net-list查看：

root@server01:~# ip netns list
qdhcp-d7614716-b1a1-43e3-b69c-4c6d2310c9b2 (id: 1)
qdhcp-2c2db99a-a8d9-428e-b51d-903fe4aa8608 (id: 0)

查看DHCP网络配置：

root@server01:~# ip netns exec qdhcp-d7614716-b1a1-43e3-b69c-4c6d2310c9b2 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ns-04037412-b9@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:be:d6:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.2/24 brd 192.168.1.255 scope global ns-04037412-b9
       valid_lft forever preferred_lft forever
    inet 169.254.169.254/16 brd 169.254.255.255 scope global ns-04037412-b9
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:febe:d6e4/64 scope link 
       valid_lft forever preferred_lft forever

 

Router配置

配置文件 /etc/neutron/l3_agent.ini

//linux bridge
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

//open vswitch
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

在Horizon创建Router完成后可以查看

root@server01:~# ip netns list
qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e (id: 2)
qdhcp-d7614716-b1a1-43e3-b69c-4c6d2310c9b2 (id: 1)
qdhcp-2c2db99a-a8d9-428e-b51d-903fe4aa8608 (id: 0)

在Router上绑定两个网段，设置外部网络（flat network）,设置了外部网络就自动实现了nat功能。

查看Router详细信息

root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: qr-92316d98-3b@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:df:ce:8e brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.1/24 brd 192.168.1.255 scope global qr-92316d98-3b
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fedf:ce8e/64 scope link 
       valid_lft forever preferred_lft forever
4: qr-cff6188b-33@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:ea:02:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.2.1/24 brd 192.168.2.255 scope global qr-cff6188b-33
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feea:22c/64 scope link 
       valid_lft forever preferred_lft forever
5: qg-29f50f19-9e@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:1c:b7:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.20.200/24 brd 192.168.20.255 scope global qg-29f50f19-9e
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe1c:b7e3/64 scope link 
       valid_lft forever preferred_lft forever

 查看Router路由表

root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.20.1    0.0.0.0         UG    0      0        0 qg-29f50f19-9e
192.168.1.0     *               255.255.255.0   U     0      0        0 qr-92316d98-3b
192.168.2.0     *               255.255.255.0   U     0      0        0 qr-cff6188b-33
192.168.20.0    *               255.255.255.0   U     0      0        0 qg-29f50f19-9e
root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e ip r
default via 192.168.20.1 dev qg-29f50f19-9e 
192.168.1.0/24 dev qr-92316d98-3b  proto kernel  scope link  src 192.168.1.1 
192.168.2.0/24 dev qr-cff6188b-33  proto kernel  scope link  src 192.168.2.1 
192.168.20.0/24 dev qg-29f50f19-9e  proto kernel  scope link  src 192.168.20.200

查看Router NAT规则

root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e iptables -t nat -L

 

分配Floating IP

1、首先需要创建一个external类型的flat网络

2、路由器设置网关为该flat网络

3、为Instance分配浮动IP

 

4、查看Router接口信息，可以看到在5:qg-29f50f19-9e@if25处已经绑定了三个192.168.20.0的IP地址

root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: qr-92316d98-3b@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:df:ce:8e brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.1.1/24 brd 192.168.1.255 scope global qr-92316d98-3b
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fedf:ce8e/64 scope link 
       valid_lft forever preferred_lft forever
4: qr-cff6188b-33@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:ea:02:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.2.1/24 brd 192.168.2.255 scope global qr-cff6188b-33
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feea:22c/64 scope link 
       valid_lft forever preferred_lft forever
5: qg-29f50f19-9e@if25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:1c:b7:e3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.20.200/24 brd 192.168.20.255 scope global qg-29f50f19-9e
       valid_lft forever preferred_lft forever
    inet 192.168.20.192/32 brd 192.168.20.192 scope global qg-29f50f19-9e
       valid_lft forever preferred_lft forever
    inet 192.168.20.201/32 brd 192.168.20.201 scope global qg-29f50f19-9e
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fe1c:b7e3/64 scope link 
       valid_lft forever preferred_lft forever

5、查看Router NAT表

root@server01:~# ip netns exec qrouter-df0c9fa9-c6d8-4b59-91d2-59917858228e iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
neutron-l3-agent-PREROUTING  all  --  anywhere             anywhere            

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
neutron-l3-agent-OUTPUT  all  --  anywhere             anywhere            

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
neutron-l3-agent-POSTROUTING  all  --  anywhere             anywhere            
neutron-postrouting-bottom  all  --  anywhere             anywhere            

Chain neutron-l3-agent-OUTPUT (1 references)
target     prot opt source               destination         
DNAT       all  --  anywhere             192.168.20.192       to:192.168.2.18
DNAT       all  --  anywhere             192.168.20.201       to:192.168.1.5

Chain neutron-l3-agent-POSTROUTING (1 references)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             ! ctstate DNAT

Chain neutron-l3-agent-PREROUTING (1 references)
target     prot opt source               destination         
REDIRECT   tcp  --  anywhere             169.254.169.254      tcp dpt:http redir ports 9697
DNAT       all  --  anywhere             192.168.20.192       to:192.168.2.18
DNAT       all  --  anywhere             192.168.20.201       to:192.168.1.5

Chain neutron-l3-agent-float-snat (1 references)
target     prot opt source               destination         
SNAT       all  --  192.168.2.18         anywhere             to:192.168.20.192
SNAT       all  --  192.168.1.5          anywhere             to:192.168.20.201

Chain neutron-l3-agent-snat (1 references)
target     prot opt source               destination         
neutron-l3-agent-float-snat  all  --  anywhere             anywhere            
SNAT       all  --  anywhere             anywhere             to:192.168.20.200
SNAT       all  --  anywhere             anywhere             mark match ! 0x2/0xffff ctstate DNAT to:192.168.20.200

Chain neutron-postrouting-bottom (1 references)
target     prot opt source               destination         
neutron-l3-agent-snat  all  --  anywhere             anywhere             /* Perform source NAT on outgoing traffic. */

 

VXLAN Network配置

相关概念：

1、支持1677W vxlan id

2、避免MAC表耗尽

3、vxlan数据包封装在UDP中，天然放环，并且支持等价多路径

4、VTEP （vxlan tunnel endpoint） 每个neutron l2 agent都需要配置一个vtep地址，该地址用作vxlan数据包的封装与解封

5、vlan端口 UDP 8472 (实际上就是在每个neurton l2 agent上启动了一个 UDP的Socket 来监听8472端口)

6、启用vxlan的同时，需要启用l2population

 

首先，VTEP地址需要手动配置到相应的物理网卡上。

配置文件：

 /etc/neutron/plugins/ml2/ml2_config.ini

[ml2]
type_drivers = local,flat,vlan,vxlan
tenant_network_types = vxlan
mechanism_drivers = linuxbridge,l2population
extension_drivers = port_security

[ml2_type_vxlan]
vni_ranges = 1:1000

 /etc/neutron/plugins/ml2/linuxbridge_agent.ini

[vxlan]
enable_vxlan = true
local_ip = 192.168.20.180
l2_population = True
arp_responder = True

配置完成后，重启Neutron相关服务，

然后在Darshboard上创建一个vxlan100

在neutron server上查看网络底层，自动创建了一个网桥，一个vxlan100,一个tap（DHCP）

root@server01:/etc/neutron/plugins/ml2# brctl show
bridge name	bridge id		STP enabled	interfaces
brqb305e6bb-b8		8000.16af07ec2857	no		tapa9792d82-06
							vxlan-100
virbr0		8000.5254000b949d	yes		virbr0-nic

 查看vlan100的详细信息, 这里可以看到 vxlan id 100, dev eth1, 因为配置了l2population ，这里有一个 proxy ageing 300

root@server01:~# ip -d link show dev vxlan-100
32: vxlan-100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master brqb305e6bb-b8 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether 16:af:07:ec:28:57 brd ff:ff:ff:ff:ff:ff promiscuity 1 
    vxlan id 100 dev eth1 srcport 0 0 dstport 8472 proxy ageing 300 
    bridge_slave state forwarding priority 32 cost 100 hairpin off guard off root_block off fastleave off learning on flood on addrgenmode eui64

查看vxlan100的 forwarding database, 这里可以看到有两个mac地址绑定的ip为VTEP 192.168.100.13

root@server01:~# bridge fdb show dev vxlan-100
fa:16:3e:95:8c:b3 master brqb305e6bb-b8 
16:af:07:ec:28:57 vlan 1 master brqb305e6bb-b8 permanent
16:af:07:ec:28:57 master brqb305e6bb-b8 permanent
00:00:00:00:00:00 dst 192.168.100.13 self permanent
fa:16:3e:95:8c:b3 dst 192.168.100.13 self permanent
fa:16:3e:f7:b1:d2 dst 192.168.100.13 self permanent

 

Firewall as a Service

这里需要注意的是Firewall as a Service是配置在Router上的，而安全组是配置在LinuxBridge上的。

也就是说：安全组保护的是 instance。

也就是说：FWaaS 保护的是 subnet。

因为 FWaaS 是在 router 中实现的，所以 FWaaS 没有单独的 agent。

启用安全组

linuxbridge_agent.ini或openvswitch_agent.ini

[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver　

配置文件

1、/etc/neutron/fwaas_driver.ini(这里的IptablesFwaasDriver是iptables_fwaas的一个类，保存在/usr/lib/python2.7/dist-packages/neutron_fwaas/services/firewall/drivers/linux/)

[fwaas]
driver = neutron_fwaas.services.firewall.drivers.linux.iptables_fwaas.IptablesFwaasDriver
enabled = true

2、/etc/neutron/neutron.conf(同样FirewallPlugin是fwaas_plugin的一个类)

[default]
service_plugins =router,neutron_fwaas.services.firewall.fwaas_plugin.FirewallPlugin　　

重启服务

/etc/init.d/neutron-server restart    /etc/init.d/neutron-l3-agent restart　

查看结果

FWaaS 有三个重要概念： Firewall、Policy 和 Rule。

Rule(首先创建rule)
Rule 是访问控制的规则，由源与目的子网 IP、源与目的端口、协议、allow 或 deny 动作组成。

Policy(第二部创建policy并关联rule)
Policy 是 Rule 的集合，Firewall 会按顺序应用 Policy 中的每一条 Rule。

Firewall(第三步创建firewall关联policy,应用到Router)
租户能够创建和管理的逻辑防火墙资源。 Firewall 必须关联某个 Policy，因此必须先创建 Policy。

 

posted @ 2017-04-27 17:20  Vincen_shen  阅读(3396)  评论(0)    收藏  举报