Floating IP in OpenStack Neutron

  • 前言

Floating IP 是相对于Fixed IP而言的,它一般是在VM创建后分配给VM的,可以达到的目的就是,外界可以访问通过这个Floating Ip访问这个VM,VM也可以通过这个IP访问外界。

在OpenStack中,这个Floating IP使用了namespace内的iptables建立NAT 转发机制来达到VM与外界的通讯的。这片文章主要讲述如何使用OpenStack搭建和使用Floating IP.

  • Environment Setup
  1.  Ubuntu 14.04 LTS
  2. 2个网卡,分别是eth0(192.168.1.46) 和 eth1(192.168.2.46,这个interface在switch使用vlanid=100)
  3. 如果你的机器是VM,请参照http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004099,去开启eth0 和eth1的promisic模式, 这一步很关键,如果没有启用,外界是无法ping通分配给VM的Floating IP的
  • Devstack Configuration
  1. 启用Nova, Neutron, Cinder, Keystone, Glance
  2. 安装OpenvSwitch,然后做如下配置
  3. # Add eth1 into br-eth1 and set eth0 and eth1 to promisc mode
    sudo ovs-vsctl br-exists br-eth1 || sudo ovs-vsctl add-br br-eth1
    sudo ovs-vsctl --may-exist add-port br-eth1 eth1
    sudo ip link set dev eth1 promisc on
    sudo ip link set dev eth0 promisc on
    sudo ip addr flush eth1
    sudo ip link set dev eth1 up
  • Devstack local.conf
  • #  Note: please change XX to your devstack node IP,  change VLAN_START and VLAN_END to your reserved vlan range Here
    # 我的IP是192.168.1.46, 所在网络是192.168.1.0/24
    [[local|localrc]]
    # Set API endpoint host using HOST_IP
    HOST_IP=XX
    # Use to specify the endpoint
    SERVICE_HOST=XX
    
    ADMIN_PASSWORD=welcome
    MYSQL_PASSWORD=welcome
    RABBIT_PASSWORD=welcome
    SERVICE_PASSWORD=welcome
    SERVICE_TOKEN=welcome
    disable_service h-eng
    disable_service h-api
    disable_service h-api-cfn
    disable_service h-api-cw
    disable_service tempest
    #enable_service tempest
    disable_service dstat
    # Enable Neturon
    disable_service n-net
    enable_service q-svc
    enable_service q-agt
    enable_service q-dhcp
    enable_service q-l3
    enable_service q-meta
    enable_service neutron
    
    # stack.sh will freshen each repo on each run if RECLONE
    # is set to yes
    RECLONE=True
    
    # Setting OFFLINE=True to enable stack.sh to run multiple
    # times without an Internet connection
    OFFLINE=False
    
    # Set FLAT_INTERFACE to the Ethernet interface that connects
    # the host to your local network
    FLAT_INTERFACE=eth1
    #FLOATING_RANGE=192.168.1.0/24
    #Q_FLOATING_ALLOCATION_POOL=start=192.168.1.100,end=192.168.1.119
    #PUBLIC_NETWORK_GATEWAY=192.168.1.30
    PUBLIC_INTERFACE=eth0
    PHYSICAL_NETWORK=public_eth1
    OVS_PHYSICAL_BRIDGE=br-ex
    #PUBLIC_BRIDGE=br-ex
    
    FIXED_RANGE=192.168.100.0/24
    #FIXED_NETWORK_SIZE=32
    NETWORK_GATEWAY=192.168.100.1
    
    
    # IMAGE_URLS accepts a comma separated list of images to pre-load into OpenStack
    IMAGE_URLS=http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
    [[post-config|$NOVA_CONF]]
    [DEFAULT]
    rpc_response_timeout=300
    service_down_time=300
    libvirt_iscsi_use_multipath = True
    #force_config_drive = False
    
    [libvirt]
    iscsi_use_multipath = True
    
    [database]
    max_pool_size=40
    max_overflow=60
    
    [[post-config|/$Q_PLUGIN_CONF_FILE]]
    [ml2]
    tenant_network_types = vlan,flat
    [ml2_type_flat]
    flat_networks = public_eth0
    [ml2_type_vlan]
    network_vlan_ranges = public_eth1:VLAN_START:VLAN_END
    
    [ovs]
    bridge_mappings = public_eth0:br-ex,public_eth1:br-eth1
    enable_tunneling = False

     

  • Setup flat and vlan network​
  1. 接下来,我要做两件事,一个是创建一个VLAN ID为100的private network,以后的虚拟机默认是从这个网络分配IP的。
  2. 然后是一个flat的public network,这个网络上是用来分配floating IP

创建private network,VLAN ID为100

stack@openstack-wangp11-01:~/devstack$ neutron net-create --provider:network_type vlan  --provider:physical_network public_eth1 --provider:segmentation_id 100 vlan_100​
stack@openstack-wangp11-01:~/devstack$ neutron net-show vlan_100
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| id                        | 2426ff0d-953f-467f-a564-c4f63d926836 |
| mtu                       | 0                                    |
| name                      | vlan_100                             |
| port_security_enabled     | True                                 |
| provider:network_type     | vlan                                 |
| provider:physical_network | public_eth1                          |
| provider:segmentation_id  | 100                                  |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f |
| tenant_id                 | 8cb8c084ffb84914b41d5044ecbcad4e     |
+---------------------------+--------------------------------------+
stack@openstack-wangp11-01:~/devstack$ neutron subnet-create --enable-dhcp  --ip-version 4 --name vlan_100_subnet01 2426ff0d-953f-467f-a564-c4f63d926836  192.168.46.0/24​
stack@openstack-wangp11-01:~/devstack$ neutron subnet-show fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
+-------------------+----------------------------------------------------+
| Field             | Value                                              |
+-------------------+----------------------------------------------------+
| allocation_pools  | {"start": "192.168.46.2", "end": "192.168.46.254"} |
| cidr              | 192.168.46.0/24                                    |
| dns_nameservers   |                                                    |
| enable_dhcp       | True                                               |
| gateway_ip        | 192.168.46.1                                       |
| host_routes       |                                                    |
| id                | fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f               |
| ip_version        | 4                                                  |
| ipv6_address_mode |                                                    |
| ipv6_ra_mode      |                                                    |
| name              | vlan_100_subnet01                                  |
| network_id        | 2426ff0d-953f-467f-a564-c4f63d926836               |
| subnetpool_id     |                                                    |
| tenant_id         | 8cb8c084ffb84914b41d5044ecbcad4e                   |
+-------------------+----------------------------------------------------+

接下来,创建Floating IP所在的public network, 网络类型是flat

注意下面:192.168.1.100-192.168.1.119是网络管理员分配给我的IP pool,192.168.1.30是我们网络的router ip

# Step 1: create external flat network
$ neutron net-create --router:external  --provider:network_type flat  --provider:physical_network public_eth0  public_net
# Step 2: create subnet based on your reservation Here
$ neutron subnet-create  --name public_eth0_subnet01 --allocation-pool start=192.168.1.100,end=192.168.1.119 --gateway​ 192.168.1.30 --disable-dhcp --ip-version 4  22d01ce7-b4c4-4af2-bc3c-9c3991903b4c 192.168.1.0/24
# Step 3: create a external router 
$ neutron router-create  router1
# Step 4: connect router1 with the private subnet via 'neutron router-interface-add <router-id> <private_subnet_id>'
$ neutron router-interface-add 2da62299-6e41-4b24-a1c7-a3d6cc4db1c8 fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f
# Step 5: Set the external public network  as the router gateway 'neutron router-gateway-set <router-id> <public_subnet_id>'
$ neutron router-gateway-set   73a38db4-20f9-49ba-b855-472be1e2fd45  e0368da6-197e-4c46-bfd1-4897d61b519a
# Step 6: Create floating IP from public network 'neutron floatingip-create  <public network id>'
$ neutron floatingip-create  1452712a-acb3-4341-a5ca-1d838eb2feb9
# Step 7: assicate the VM instance's port in private network to the newly created floating ip port
## get the port id of floating IP (in bold)
$ stack@openstack-wangp11-01:~$ neutron floatingip-list
+--------------------------------------+------------------+---------------------+--------------------------------------+
| id                                   | fixed_ip_address | floating_ip_address | port_id                              |
+--------------------------------------+------------------+---------------------+--------------------------------------+
| a2316e0b-6d72-420f-8a23-f8421160d3d4 |                | 192.168.1.102       | 75353e16-5a16-452e-b420-0ada719d625c |
+--------------------------------------+------------------+---------------------+--------------------------------------+
## get the port id of private interface attached to VM instance
$ stack@openstack-wangp11-01:~$ neutron port-list
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                            |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| 16a3e39c-48ef-4d4d-bc8a-96b27da7ff3c |      | fa:16:3e:c1:24:51 | {"subnet_id": "e0368da6-197e-4c46-bfd1-4897d61b519a", "ip_address": "192.168.1.100"} |
| 616d03cd-df6a-465f-a23a-b6bf46e55f7a |      | fa:16:3e:48:2e:7f | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.4"}  |
| 713e4541-7d22-4403-b828-94d714d94ec0 |      | fa:16:3e:af:43:7b | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.2"}  |
| 75353e16-5a16-452e-b420-0ada719d625c |      | fa:16:3e:23:11:db | {"subnet_id": "fab94e2f-f7de-4bf6-8e9b-7ade0fcfad1f", "ip_address": "192.168.46.3"} 

# 上面红色的port是VM在private network的interface $ stack@openstack
-wangp11-01:~$ neutron floatingip-associate a2316e0b-6d72-420f-8a23-f8421160d3d4 75353e16-5a16-452e-b420-0ada719d625c # Now you can use '192.168.1.102' to login to the VM instance from 192.168.1.XX network

我在最前面提到,这个Floating IP是通过namespace内的iptables规则实现的,如何查看呢?下面是在neutron node上的一个实例,192.168.1.102是Floating IP,192.168.46.3是private IP

ip netns exec qrouter-<private-network-id> iptables -t nat -S

-A quantum-l3-agent-OUTPUT -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3
-A quantum-l3-agent-PREROUTING -d 192.168.1.102/32 -j DNAT --to-destination 192.168.46.3 
-A quantum-l3-agent-float-snat -s 192.168.46.3/32 -j SNAT --to-source 192.168.1.102
  • 启用nameserver

现在,VM跟外界可以通过192.168.1.102进行通信了,在外界看来,VM的Ip就是192.168.1.102了,但是有一个问题,无法访问网站,也无法sudo apt-get update

必须要启用nameserver才可以访问外面的网址

# add following to /etc/resolv.conf of the VM instance to enable name service
 
nameserver <your name sever ip>
nameserver <your name server ip 2>

#and run following to enable
sudo resolvconf
-u
  • 问题诊断

TODO

  • 参考文章

https://www.mirantis.com/blog/configuring-floating-ip-addresses-networking-openstack-public-private-clouds/

https://www.rdoproject.org/Networking_in_too_much_detail

https://blogs.oracle.com/ronen/entry/running_openstack_icehouse_with_zfs

 

posted @ 2015-09-22 23:54  孤独的居士  阅读(...)  评论(... 编辑 收藏