Experience with Ubuntu/Openstack(neutron) installation.

Abstract

In this post I will write about my experience with Ubuntu, Openstack installation. Because it was an experimental installation (without any knowledge how to set it up correctly and what pet peeves are available) the style of this post will be casual. I will mention here everything from my installation: difficulties, ideas, self questions and of course documentation steps for each technology mentioned above.

Prerequisites

My task was to setup Ubuntu, Openstack on big server machine.

Installing Ubuntu

So, I've received the machine with the following hardware parameters:

CPU: 6 cores;

Memory: 16GB;

Hard-Drive: 3TB.

And I decided to install on this machine Ubuntu server 14.04.1 x64. So during the installation I had three problems:

• Ubuntu was trying to find cdrom even if you don't have a cdrom on your PC. To solve it there is a trick of mounting you usb device to /cdrom folder. But if you wait for some seconds this message will go away and you can continue installation; 
• Installation program tried to mount floppy disk but if you don't have it on your PC then you should wait for some seconds for the installation to continue. Also there are a lot of solutions available online and you can stop floppy disk mounting process;
• Partition table. I just completed same installation steps as for the desktop Ubuntu version. So everything was fine. Installation went well but after I rebooted my machine I saw a message: 

attempt to read or write outside of disk 'hd0':

Then I started to google in order to find where the mistake was during the installation. On the internet I found several solutions:

• Setting partitions manually via grub rescue;
• Reinstalling the whole system;
• "Heavy" articles about installing ubuntu server on a machine which has HDD more than 500GB. 

After reading all these articles I've concluded that I have to check my grub2 settings at the last step of my installation, and I have to repartition my hard drive in another style. So I headed into GRUB 2, read through it and repartioned my HDD to meet the requirements of grub2. The point is here: 

If you format the whole drive during Ubuntu installation then you must partition it in order not to have mistakes mentioned up here in the text. For that I created first partition of the 1MB size as a bios_grub, next one as an ext4 size preferably more than 50 GB and set it as a / mount point, and rest of HDD just mark as ext4 /home directory:

Pic.1

After that I continued installation and it was successful, Ubuntu 14.04.1 was ready for the use.

Then I launched ubuntu and I installed minimal version of lubuntu-desktop on my server. I did that because I needed GUI for my system:

# apt-get install --no-install-recommends lubuntu-desktop

And finally I've got a clean and light server running on my big machine.

Installing Openstack

My general task was to have virtual machines running Openstack. I decided to try my installation on virtual machines powered by KVM. So, I've just followed this Ubuntu KVM installation to setup virtual environment. After kvm was installed I have created virtual network through the virt-manager:

• Network name: openstack;
• Network: 10.0.0.0/24;
• DHCP: disabled;
• Forwarding to physical network (Destination): Any physical device;
• Forwarding to physical network (Mode): NAT.

Also I created 3 virtual machines with virt-manager:

Controller:

CPU: 1

Mem: 2 GB

HDD: 8 GB
NIC: 1 network card

Network: openstack (machine IP: 10.0.0.11)
OS: Ubuntu image server 14.04.2

Network:

CPU: 1

Mem: 512 MB

HDD: 8 GB
NIC: 2 network card

Network: openstack (machine IP: 10.0.0.21)

OS: Ubuntu image server 14.04.2

Compute1:

CPU: 1

Mem: 2 GB

HDD: 10 GB
NIC: 1 network card

Network: openstack (machine IP: 10.0.0.31)

OS: Ubuntu image server 14.04.2

After creating these virtual machines I installed Ubuntu 14.04.2 on them. During the installation:

1). Gave machine a hostname as its role in Openstack;

2). In the installation chose only OpenSSH server;
3). Disabled automatic updates.

After system was installed on all virtual machines then I started Openstack installation:

1). # apt-get update

2). # apt-get dist-upgrade

3). Then I followed all the steps in Openstack Juno (neutron)

So, first step Basic environment is the most important one. And I am going to talk about it. Please be more attentive with this part. I had following interfaces for controller, network and compute nodes (these configurations are wrong, it is just an example of what I had at the beginning):

controller_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

# The management interface
auto eth0:0
iface eth0:0 inet static
           address 10.0.0.11
           netmask 255.255.255.0
           gateway 10.0.0.1

network_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The management interface
auto eth0:0
iface eth0:0 inet static
           address 10.0.0.21
           netmask 255.255.255.0
           gateway 10.0.0.1

# The instance tunnels interface
auto eth0:1
iface eth0:1 inet static
           address 10.0.1.21
           netmask 255.255.255.0

# The external network interface
auto eth0
iface eth0 inet manual
           up ip link set dev $IFACE up
           down ip link set dev $IFACE down

compute_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

# The management interface
auto eth0:0
iface eth0:0 inet static
           address 10.0.0.31
           netmask 255.255.255.0
           gateway 10.0.0.1

# The instance tunnels interface
auto eth0:1
iface eth0:1 inet static
           address 10.0.1.31
           netmask 255.255.255.0

So, then I followed the openstack documentation and when I reached the part of adding openvswitch bridge on eth0 interface of a network node I lost connection to it. I couldn't establish ssh connection to the network node. So then I started to look for a solution and luckily I found this awesome video about Openvswitch:

It explains basics of openvswitch. And here question raises "How to setup network interfaces correctly for a virtual machine?" So, looking for better explanations on networking for Openstack I found something interesting:

The external network from the network node is used to connect instances to the outside world. It has no IP because it uses a special configuration to run in promiscuous mode. Basically, what it means, is that that network adapter will accept any kind of package. In normal mode, if a package is received and is not meant for that network interface(has a different MAC, etc), the package is dropped. In promiscuous mode all packages are received. Because all instances will use that network adapter to connect to the internet, that ethernet has to accept multiple and different packages. The management network is used for all communications between nodes, data, services etc. This network should be connected to the internet but only for outgoing connection. You should close all incoming ports for safety reasons. The instance tunnel is used for communication between instances and network node. Hope this helps!

Well, then each machine will have at least one automatic eth0 interface through which we will access machines via ssh and manipulate machines, other interfaces will be created through setup. So you should count on how much NICs you have. The best case is to have:

• controller_node: 1 NIC;
• network_node:   2 NICs;
• compute_node:  1 NIC.

We need these number of NICs because if we look at openstack neutron architecture:

we can see that network, compute nodes have more than 1 interface. So, also I found some good information in Openstack Icehouse documentation:

The host (network node) must have an IP address associated with an interface other than EXTERNAL_INTERFACE, and your remote terminal session must be associated with this other IP address.

If you associate an IP address with EXTERNAL_INTERFACE, that IP address stops working after you issue the ovs-vsctl add-port br-ex EXTERNAL_INTERFACE command. If you associate a remote terminal session with that IP address, you lose connectivity with the host. For more details about this behavior, see the Configuration Problems section of the Open vSwitch FAQ.

Then I reconfigured /etc/network/interfaces of virtual machines which I was using:

controller_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The management network interface
auto eth0
iface eth0 inet static
        address 10.0.0.11
        netmask 255.255.255.0
        gateway 10.0.0.1

network_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The management network interface
auto eth0
iface eth0 inet static
        address 10.0.0.21
        netmask 255.255.255.0
        gateway 10.0.0.1

# The instance tunnels interface
auto eth0:0
iface eth0:0 inet static
        address 10.0.1.21
        netmask 255.255.255.0

# The external network interface
auto eth1
iface eth1 inet manual
        up ip link set dev $IFACE up
        down ip link set dev $IFACE down

compute_node_interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The management network interface
auto eth0
iface eth0 inet static
        address 10.0.0.31
        netmask 255.255.255.0
        gateway 10.0.0.1

# The instance tunnels interface
auto eth0:0
iface eth0:0 inet static
        address 10.0.1.31
        netmask 255.255.255.0

After that I continued installation and Openstack was ready to run (I could launch vm-instance via VNC). So, success!