I managed to prepare a two-node OpenStack Havana setup, hand-configured (URL to notes below). Here are some Neutron configurations that worked for me.
Setup details:
- Two Fedora 20 minimal (@core) virtual machines to run the Controller & Compute nodes.
- Services on Controller node: Keystone, Cinder, Glance, Neutron, Nova. Neutron networking is setup with OpenvSwitch plugin, network namespaces, GRE tunneling.
- Services on Compute node: Nova (openstack-nova-compute service), Neutron (neutron-openvswitch-agent), libvirtd, OpenvSwitch.
- Both nodes are manually configured. Notes is here.
Configurations
OpenvSwitch plugin configuration — /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini — on Controller node:
$ cat plugin.ini | grep -v ^$ | grep -v ^# [ovs] [agent] [securitygroup] [ovs] tenant_network_type = gre tunnel_id_ranges = 1:1000 enable_tunneling = True integration_bridge = br-int tunnel_bridge = br-tun local_ip = 192.168.122.163 [DATABASE] sql_connection = mysql://neutron:fedora@vm01-controller/ovs_neutron [SECURITYGROUP] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
Neutron configuration — /etc/neutron/neutron.conf:
$ cat neutron.conf | grep -v ^$ | grep -v ^# [DEFAULT] core_plugin = neutron.plugins.openvswitch.ovs_neutron_plugin.OVSNeutronPluginV2 rpc_backend = neutron.openstack.common.rpc.impl_qpid qpid_hostname = localhost auth_strategy = keystone ovs_use_veth = True allow_overlapping_ips = True qpid_port = 5672 [quotas] quota_network = 20 quota_subnet = 20 [agent] root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf [keystone_authtoken] auth_host = 192.168.122.163 admin_tenant_name = services admin_user = neutron admin_password = fedora [database] [service_providers]
Neutron L3 agent configuration — /etc/neutron/neutron.conf:
$ cat l3_agent.ini | grep -v ^$ | grep -v ^# [DEFAULT] interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver handle_internal_only_routers = TRUE ovs_use_veth = True use_namespaces = True metadata_ip = 192.168.122.163 metadata_port = 8700
Neutron metadata agent — /etc/neutron/metadata_agent.ini:
$ cat metadata_agent.ini | grep -v ^$ | grep -v ^# [DEFAULT] auth_url = http://192.168.122.163:35357/v2.0/ auth_region = regionOne admin_tenant_name = services admin_user = neutron admin_password = fedora nova_metadata_ip = 192.168.122.163 nova_metadata_port = 8700 metadata_proxy_shared_secret = fedora
iptables rules on Controller node:
$ cat /etc/sysconfig/iptables *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -p tcp -m multiport --dports 3260 -m comment --comment "001 cinder incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 80 -m comment --comment "001 horizon incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9292 -m comment --comment "001 glance incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5000,35357 -m comment --comment "001 keystone incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 3306 -m comment --comment "001 mariadb incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 6080 -m comment --comment "001 novncproxy incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8770:8780 -m comment --comment "001 novaapi incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 9696 -m comment --comment "001 neutron incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 5672 -m comment --comment "001 qpid incoming" -j ACCEPT -A INPUT -p tcp -m multiport --dports 8700 -m comment --comment "001 metadata incoming" -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5999 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A INPUT -p gre -j ACCEPT -A OUTPUT -p gre -j ACCEPT -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT
iptables rules on Compute node:
$ cat /etc/sysconfig/iptables *filter :INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5999 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT
OpenvSwitch database contents:
$ ovs-vsctl show
6f5d0e33-7013-4816-bc97-29af9abe8309
Bridge br-int
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Port "tap63ea2815-b5"
tag: 1
Interface "tap63ea2815-b5"
Bridge br-ex
Port "eth0"
Interface "eth0"
Port "tape7110dba-a9"
Interface "tape7110dba-a9"
Port br-ex
Interface br-ex
type: internal
Bridge br-tun
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "gre-2"
Interface "gre-2"
type: gre
options: {in_key=flow, local_ip="192.168.122.163", out_key=flow, remote_ip="192.168.122.100"}
ovs_version: "2.0.0"
NOTE: I SCPed the Neutron configurations neutron.conf and OpenvSwitch plugin plugin.ini from Controller to Compute node (don’t miss to replace local_ip attribute appropriately — I made that mistake).
A couple of non-deterministic issues I’m still investigating on a new setup with a non-default libvirt network as external network (on my current setup I used libvirt’s default subnet (192.168.x.x.). Lars pointed out that could probably be the cause of some of the routing issues):
- Sporadic loss of networking for Nova guests. This got resolved (at-least partially) when I invoke VNC of the guest (via SSH tunneling) & do some basic diagnostics, networking comes up just fine in the guests again (GRE tunnels go stale?).
tcmpdumpanalysis on various network devices (tunnels/bridges/tap devices) on both Controller & Compute nodes in progress. - Nova guests fail to acquire DHCP leases (I can clearly observe this, when I explicitly do an
ifdown eth0 && ifup eth0from VNC of the guest. Neutron DHCP agent seems to be flaky here.
TIP: On Fedora, openstack-utils package(from version: openstack-utils-2013.2-2.fc21.noarch) includes a neat utility called openstack-service which allows to trivially control OpenStack services. This makes life much easier, thanks to Lars!
Kashyap Chamarthy 