Openstack with Kolla on CentOS 7

Topology

I am going to build a multinode Openstack environment. I use an HP DL320e with Core a i3 and 16GB of memory as my controller node. The minimum requirements for running a controller is 8GB of memory and trust me, it uses it all! I use an HP DL320e with a E3-1241v3 and 32GB of memory as my compute node. As deploy node on which Kolla software is installed I use a VM with 2GB memory and 4 vCPUs. I am going to use my Synology as NFS storage. Openstack uses NFSv4 so it really does not matter what kind of NAS you use.

Base installation

Use my Docker host on CentOS 7 howto on every host.

Disable SELinux

I hate to disable SElinux but SELinux and Kolla is a work in progress. At the moment it does not work very well, so we have to disable it the hosts we are going to deploy it on. In this case not on the Kolla VM.

/etc/selinux/config

.
.
SELINUX=disabled
.
.

Disable Firewalld

For the moment I am going to disable firewalld because I do not know the used ports yet….

systemctl disable firewalld

Docker repository

The Kolla server needs to be a Docker repository for all the Docker containers.

yum -y install docker-distribution

Create SSL key and csr and sign it using your own CA. Put them in the place defined in the config.yml

/etc/docker-distribution/registry/config.yml

.
.
http:
    addr: :5000
    tls:
        certificate: "/etc/pki/tls/certs/kolla.mngt.bh.helux.nl.crt"
        key: "/etc/pki/tls/private/kolla.mngt.bh.helux.nl.key"

Create an LV for the repository

lvcreate -L16G -n registry /dev/rootvg
mkfs.ext4 -m0 /dev/rootvg/registry

/etc/fstab

.
.
/dev/mapper/rootvg-docker /var/lib/registry ext4 defaults,noatime,discard 1 2

mount -a

systemctl start docker-distribution
systemctl enable docker-distribution.service

Test the connection on the other hosts. Output should be:

curl https://kolla.mngt.bh.helux.nl:5000/v2

<a href="/v2/">Moved Permanently</a>.

Prerequisites for Kolla

yum -y install python-pip python-docker-py ansible python-devel libffi-devel gcc openssl-devel

pip install -U pip

pip install -U python-openstackclient python-neutronclient python-designateclient

Install Kolla

pip install kolla
pip install kolla-ansible

cp -r /usr/share/kolla-ansible/etc_examples/kolla /etc/

Configure Kolla

Compare the original with my globals.yml and edit to your needs.

/etc/kolla/globals.yml

---
# You can use this file to override _any_ variable throughout Kolla.
# Additional options can be found in the
# 'kolla-ansible/ansible/group_vars/all.yml' file. Default value of all the
# commented parameters are shown here, To override the default value uncomment
# the parameter and change its value.

###################
# Kolla options
###################
# Valid options are [ COPY_ONCE, COPY_ALWAYS ]
config_strategy: "COPY_ALWAYS"

# Valid options are [ centos, oraclelinux, ubuntu ]
kolla_base_distro: "centos"

# Valid options are [ binary, source ]
kolla_install_type: "binary"

# Valid option is Docker repository tag
#openstack_release: "auto"

# Location of configuration overrides
node_custom_config: "/etc/kolla/config"

# This should be a VIP, an unused IP on your network that will float between
# the hosts running keepalived for high-availability. When running an All-In-One
# without haproxy and keepalived, this should be the first IP on your
# 'network_interface' as set in the Networking section below.
kolla_internal_vip_address: "172.16.2.33"

# This is the DNS name that maps to the kolla_internal_vip_address VIP. By
# default it is the same as kolla_internal_vip_address.
#kolla_internal_fqdn: "{{ kolla_internal_vip_address }}"
kolla_internal_fqdn: "controller.mngt.bh.helux.nl"

# This should be a VIP, an unused IP on your network that will float between
# the hosts running keepalived for high-availability. It defaults to the
# kolla_internal_vip_address, allowing internal and external communication to
# share the same address.  Specify a kolla_external_vip_address to separate
# internal and external requests between two VIPs.
#kolla_external_vip_address: "{{ kolla_internal_vip_address }}"
kolla_external_vip_address: "172.16.2.35"

# The Public address used to communicate with OpenStack as set in the public_url
# for the endpoints that will be created. This DNS name should map to
# kolla_external_vip_address.
#kolla_external_fqdn: "{{ kolla_external_vip_address }}"
kolla_external_fqdn: "cloud.mngt.bh.helux.nl"

####################
# Docker options
####################
# Below is an example of a private repository with authentication. Note the
# Docker registry password can also be set in the passwords.yml file.

docker_registry: "kolla.mngt.bh.helux.nl:5000"
#docker_registry: "172.16.0.10:4000"
#docker_namespace: "companyname"
#docker_registry_username: "sam"
#docker_registry_password: "correcthorsebatterystaple"


###############################
# Neutron - Networking Options
###############################
# This interface is what all your api services will be bound to by default.
# Additionally, all vxlan/tunnel and storage network traffic will go over this
# interface by default. This interface must contain an IPv4 address.
# It is possible for hosts to have non-matching names of interfaces - these can
# be set in an inventory file per host or per group or stored separately, see
#     http://docs.ansible.com/ansible/intro_inventory.html
# Yet another way to workaround the naming problem is to create a bond for the
# interface on all hosts and give the bond name here. Similar strategy can be
# followed for other types of interfaces.
network_interface: "vlan2"

# These can be adjusted for even more customization. The default is the same as
# the 'network_interface'. These interfaces must contain an IPv4 address.
#kolla_external_vip_interface: "{{ network_interface }}"
#api_interface: "{{ network_interface }}"
#storage_interface: "{{ network_interface }}"
#cluster_interface: "{{ network_interface }}"
#tunnel_interface: "{{ network_interface }}"
#dns_interface: "{{ network_interface }}"

# This is the raw interface given to neutron as its external network port. Even
# though an IP address can exist on this interface, it will be unusable in most
# configurations. It is recommended this interface not be configured with any IP
# addresses for that reason.
neutron_external_interface: "vlan11"

# Valid options are [ openvswitch, linuxbridge ]
neutron_plugin_agent: "openvswitch"


####################
# keepalived options
####################
# Arbitrary unique number from 0..255
#keepalived_virtual_router_id: "51"


####################
# TLS options
####################
# To provide encryption and authentication on the kolla_external_vip_interface,
# TLS can be enabled.  When TLS is enabled, certificates must be provided to
# allow clients to perform authentication.
#kolla_enable_tls_external: "no"
#kolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/haproxy.pem"
kolla_enable_tls_external: "yes"
kolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/haproxy.pem"
kolla_external_fqdn_cacert: "{{ node_config_directory }}/certificates/haproxy-ca.pem"


####################
# OpenStack options
####################
# Use these options to set the various log levels across all OpenStack projects
# Valid options are [ True, False ]
#openstack_logging_debug: "False"

# Valid options are [ novnc, spice ]
#nova_console: "novnc"

# OpenStack services can be enabled or disabled with these options
enable_aodh: "no"
enable_barbican: "no"
enable_ceilometer: "no"
enable_central_logging: "no"
enable_ceph: "no"
enable_ceph_rgw: "no"
enable_chrony: "yes"
enable_cinder: "yes"
enable_cinder_backend_hnas_iscsi: "no"
enable_cinder_backend_hnas_nfs: "no"
enable_cinder_backend_iscsi: "yes"
enable_cinder_backend_lvm: "no"
enable_cinder_backend_nfs: "yes"
enable_cloudkitty: "no"
enable_collectd: "no"
enable_congress: "no"
enable_designate: "yes"
enable_destroy_images: "no"
enable_etcd: "no"
enable_freezer: "no"
enable_gnocchi: "no"
enable_grafana: "no"
enable_heat: "yes"
enable_horizon: "yes"
enable_horizon_cloudkitty: "{{ enable_cloudkitty | bool }}"
enable_horizon_freezer: "{{ enable_freezer | bool }}"
enable_horizon_ironic: "{{ enable_ironic | bool }}"
enable_horizon_karbor: "{{ enable_karbor | bool }}"
enable_horizon_magnum: "{{ enable_magnum | bool }}"
enable_horizon_manila: "{{ enable_manila | bool }}"
enable_horizon_mistral: "{{ enable_mistral | bool }}"
enable_horizon_murano: "{{ enable_murano | bool }}"
enable_horizon_neutron_lbaas: "{{ enable_neutron_lbaas | bool }}"
enable_horizon_sahara: "{{ enable_sahara | bool }}"
enable_horizon_searchlight: "{{ enable_searchlight | bool }}"
enable_horizon_senlin: "{{ enable_senlin | bool }}"
enable_horizon_solum: "{{ enable_solum | bool }}"
enable_horizon_tacker: "{{ enable_tacker | bool }}"
enable_horizon_trove: "{{ enable_trove | bool }}"
enable_horizon_watcher: "{{ enable_watcher | bool }}"
enable_influxdb: "no"
enable_ironic: "no"
enable_karbor: "no"
enable_kuryr: "no"
enable_magnum: "no"
enable_manila: "no"
enable_manila_backend_generic: "no"
enable_manila_backend_hnas: "no"
enable_mistral: "no"
enable_mongodb: "no"
enable_murano: "no"
enable_multipathd: "no"
enable_neutron_dvr: "no"
enable_neutron_lbaas: "no"
enable_neutron_fwaas: "no"
enable_neutron_qos: "no"
enable_neutron_agent_ha: "no"
enable_neutron_vpnaas: "no"
enable_nova_serialconsole_proxy: "no"
enable_octavia: "no"
enable_panko: "no"
enable_rally: "no"
enable_sahara: "no"
enable_searchlight: "no"
enable_senlin: "no"
enable_solum: "no"
enable_swift: "no"
enable_telegraf: "no"
enable_tacker: "no"
enable_tempest: "no"
enable_trove: "no"
enable_vmtp: "no"
enable_watcher: "no"

###################
# Ceph options
###################
# Ceph can be setup with a caching to improve performance. To use the cache you
# must provide separate disks than those for the OSDs
#ceph_enable_cache: "no"
# Valid options are [ forward, none, writeback ]
#ceph_cache_mode: "writeback"

# A requirement for using the erasure-coded pools is you must setup a cache tier
# Valid options are [ erasure, replicated ]
#ceph_pool_type: "replicated"

# Integrate ceph rados object gateway with openstack keystone
#enable_ceph_rgw_keystone: "no"


##############################
# Keystone - Identity Options
##############################

# Valid options are [ uuid, fernet ]
keystone_token_provider: 'uuid'

# Interval to rotate fernet keys by (in seconds). Must be an interval of
# 60(1 min), 120(2 min), 180(3 min), 240(4 min), 300(5 min), 360(6 min),
# 600(10 min), 720(12 min), 900(15 min), 1200(20 min), 1800(30 min),
# 3600(1 hour), 7200(2 hour), 10800(3 hour), 14400(4 hour), 21600(6 hour),
# 28800(8 hour), 43200(12 hour), 86400(1 day), 604800(1 week).
#fernet_token_expiry: 86400


#########################
# Glance - Image Options
#########################
# Configure image backend.
#glance_backend_file: "yes"
#glance_backend_ceph: "no"

#######################
# Ceilometer options
#######################
# Valid options are [ mongodb, mysql, gnocchi ]
#ceilometer_database_type: "mongodb"

# Valid options are [ mongodb, gnocchi, panko ]
#ceilometer_event_type: "mongodb"


#######################
# Barbican options
#######################
# Valid options are [ simple_crypto, p11_crypto ]
#barbican_crypto_plugin: "simple_crypto"
#barbican_library_path: "/usr/lib/libCryptoki2_64.so"

#######################
## Panko options
#######################
# Valid options are [ mongodb, mysql ]
#panko_database_type: "mysql"

#######################
# Gnocchi options
#######################
# Valid options are [ file, ceph ]
#gnocchi_backend_storage: "{{ 'ceph' if enable_ceph|bool else 'file' }}"


#################################
# Cinder - Block Storage Options
#################################
# Enable / disable Cinder backends
#cinder_backend_ceph: "{{ enable_ceph }}"
#cinder_volume_group: "cinder-volumes"
cinder_backup_driver: "nfs"
cinder_backup_share: "172.16.2.2:/volume1/cinder/backup"
cinder_backup_mount_options_nfs: "vers=4"


#######################
# Designate options
#######################
# Valid options are [ bind9 ]
designate_backend: "bind9"
designate_ns_record: "openstack.helux.nl"

#########################
# Nova - Compute Options
#########################
#nova_backend_ceph: "{{ enable_ceph }}"


##############################
# Horizon - Dashboard Options
##############################
#horizon_backend_database: "{{ enable_murano | bool }}"


#######################################
# Manila - Shared File Systems Options
#######################################
# HNAS backend configuration
#hnas_ip:
#hnas_user:
#hnas_password:
#hnas_evs_id:
#hnas_evs_ip:
#hnas_file_system_name:

##################################
# Swift - Object Storage Options
##################################
# Swift expects block devices to be available for storage. Two types of storage
# are supported: 1 - storage device with a special partition name and filesystem
# label, 2 - unpartitioned disk  with a filesystem. The label of this filesystem
# is used to detect the disk which Swift will be using.

# Swift support two mathcing modes, valid options are [ prefix, strict ]
#swift_devices_match_mode: "strict"

# This parameter defines matching pattern: if "strict" mode was selected,
# for swift_devices_match_mode then swift_device_name should specify the name of
# the special swift partition for example: "KOLLA_SWIFT_DATA", if "prefix" mode was
# selected then swift_devices_name should specify a pattern which would match to
# filesystems' labels prepared for swift.
#swift_devices_name: "KOLLA_SWIFT_DATA"


################################################
# Tempest - The OpenStack Integration Test Suite
################################################
# following value must be set when enable tempest
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:

# tempest_image_alt_id: "{{ tempest_image_id }}"
# tempest_flavor_ref_alt_id: "{{ tempest_flavor_ref_id }}"

openstack_region_name: "Bergschenhoek"

Profiles

I do not want to build all Openstack containers, so I have created my own profile

/etc/kolla/kolla-build.conf

[DEFAULT]
output_file = etc/kolla/kolla-build.conf
namespace = kolla

[profiles]

# Infra images (list value)
#infra = ceph,cron,mariadb,haproxy,keepalived,kolla-toolbox,memcached,mongodb,openvswitch,rabbitmq,heka

# Main images (list value)
#main = cinder,ceilometer,glance,heat,horizon,keystone,neutron,nova,swift

# Aux Images (list value)
#aux = aodh,designate,gnocchi,ironic,magnum,mistral,trove,zaqar,zookeeper

# Default images (list value)
#default = cron,kolla-toolbox,glance,horizon,keystone,memcached,mariadb,neutron,nova,openvswitch,rabbitmq

# Gate images (list value)
#gate = cron,glance,haproxy,keepalived,keystone,kolla-toolbox,mariadb,memcached,neutron,nova,openvswitch,rabbitmq,heka

# Mesos images (list value)
#mesos = chronos,marathon,mesos-dns,mesos-master,mesos-slave,zookeeper

# Helux
helux = chrony,cinder,cron,designate,fluentd,glance,heat,heka,haproxy,horizon,iscsid,keepalived,keystone,kolla-toolbox,linuxbridge,mariadb,memcached,mongodb,neutron,nova,openvswitch,rabbitmq,tgtd

Passwords

Next we have to put the password framework file in place. Kolla generates passwords for you and puts them in here.

cp /usr/share/kolla-ansible/etc_examples/kolla/passwords.yml /etc/kolla

Now run the command to generate them

kolla-genpwd

Build the Kolla containers

This takes some time so run it in a screen if your connection to the server is poor.

kolla-build --base centos --type binary --profile helux --registry kolla.mngt.bh.helux.nl:5000 --push

When done. You must see something like this:

docker images

REPOSITORY                                                                   TAG                 IMAGE ID            CREATED             SIZE
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-horizon                      4.0.1               de8e3bfbcfab        4 days ago          898MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-compute                 4.0.1               33bbc2a3ee63        4 days ago          1.26GB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-keystone-fernet              4.0.1               52d0e2eb9c56        4 days ago          730MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-keystone-ssh                 4.0.1               429c5be05af5        4 days ago          750MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-compute-ironic          4.0.1               845a026f3428        4 days ago          1.17GB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-keystone                     4.0.1               dae2a06c5199        4 days ago          708MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-glance-api                   4.0.1               b10c6bf64cf8        4 days ago          847MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cinder-volume                4.0.1               78618c3a4b0a        4 days ago          974MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cinder-api                   4.0.1               aaec739afb8b        4 days ago          1.04GB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cinder-backup                4.0.1               2000dda11ab5        4 days ago          974MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cinder-scheduler             4.0.1               fa5967f4f497        4 days ago          944MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-barbican-keystone-listener   4.0.1               59cfc7081751        4 days ago          671MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-glance-registry              4.0.1               de84de0259dd        4 days ago          788MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-server               4.0.1               eaaf0c694398        4 days ago          758MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-openvswitch-agent    4.0.1               7d602480719b        4 days ago          758MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-lbaas-agent          4.0.1               9f17462f0d8b        4 days ago          760MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-vpnaas-agent         4.0.1               eab3bdd99b03        4 days ago          784MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-linuxbridge-agent    4.0.1               f9db935dff47        4 days ago          758MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-metering-agent       4.0.1               69a65e37f850        4 days ago          758MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-dhcp-agent           4.0.1               d4fb33f30c32        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-l3-agent             4.0.1               d4fb33f30c32        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-metadata-agent       4.0.1               bc78d04b2930        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-heat-api                     4.0.1               523284fa24dd        4 days ago          675MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-heat-engine                  4.0.1               84181f375cc8        4 days ago          675MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-heat-api-cfn                 4.0.1               79def53f12f5        4 days ago          675MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-ssh                     4.0.1               3a49ab0d91d8        4 days ago          754MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-conductor               4.0.1               3ddb332c5279        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-api                     4.0.1               077f540542bf        4 days ago          786MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-consoleauth             4.0.1               4cf41b2fd03d        4 days ago          735MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-keystone-base                4.0.1               7612d1b9d9c3        4 days ago          708MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-scheduler               4.0.1               2655236e1ff8        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-serialproxy             4.0.1               2f6ce0c0597a        4 days ago          735MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cinder-base                  4.0.1               9a559fcd92a6        4 days ago          944MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-barbican-base                4.0.1               ebc7dabad10b        4 days ago          650MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-placement-api           4.0.1               81e87dafe1a6        4 days ago          786MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-glance-base                  4.0.1               60cb520d3f75        4 days ago          788MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-spicehtml5proxy         4.0.1               b8943d5f83a8        4 days ago          775MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-novncproxy              4.0.1               fbb04ab4cb86        4 days ago          735MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-libvirt                 4.0.1               5951a4753e9c        4 days ago          996MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-heat-base                    4.0.1               7272eb37bf9e        4 days ago          653MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-neutron-base                 4.0.1               035289e2c157        4 days ago          734MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-kolla-toolbox                4.0.1               022a9d002382        4 days ago          731MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-nova-base                    4.0.1               0c4f2ed22463        4 days ago          712MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-fluentd                      4.0.1               cad19192b343        4 days ago          721MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-openstack-base               4.0.1               e740f5af187a        4 days ago          619MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-rabbitmq                     4.0.1               04f83ecc38f7        4 days ago          478MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-mariadb                      4.0.1               1f335a3ca51e        4 days ago          809MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-memcached                    4.0.1               cb4eda9aff15        4 days ago          419MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-openvswitch-vswitchd         4.0.1               34e294d5e6ed        4 days ago          440MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-openvswitch-db-server        4.0.1               79519cdfffd9        4 days ago          440MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-openvswitch-base             4.0.1               2ce198da8c55        4 days ago          440MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-haproxy                      4.0.1               2c2607185b3b        4 days ago          440MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-heka                         4.0.1               f534665564c1        4 days ago          472MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-cron                         4.0.1               89bc735053f9        4 days ago          418MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-mongodb                      4.0.1               535ff81f3886        4 days ago          590MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-keepalived                   4.0.1               3c77c2f7c6df        4 days ago          424MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-chrony                       4.0.1               5b2e29385465        4 days ago          419MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-iscsid                       4.0.1               1f7261697c8d        4 days ago          423MB
kolla.mngt.bh.helux.nl:5000/kolla/centos-binary-base                         4.0.1               e035f73a1eba        4 days ago          398MB
centos                                                                       7                   a8493f5f50ff        2 weeks ago         192MB

Additional configuration

Ansible templates

We need to alter an ansible template of Kolla to enable multi domains in Horizon

/usr/share/kolla-ansible/ansible/roles/horizon/templates/local_settings.j2

.
.
#OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = False
OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True
.
.

Kolla configuration directory

The ansible scripts use the /etc/kolla directory to configure customer specific parts of Openstack.

mkdir -p /etc/kolla/config/keystone/domains

Own signed certificates

mkdir /etc/kolla/certificates

Create haproxy-ca.pem containg root CA certificate and create haproxy.pem containing the private key and certificate concatenated.

ls -l /etc/kolla/certificates/
total 8
-rw-r--r-- 1 root root 1192 Jun 20 17:44 haproxy-ca.pem
-rw-r----- 1 root root 2205 Jun 20 18:18 haproxy.pem

I use NFS storage so create a file with the NFS share info.

/etc/kolla/config/nfs_shares.j2

172.16.2.2:/volume1/cinder

/etc/kolla/config/nova.conf

[DEFAULT]
cpu_allocation_ratio = 16.0
ram_allocation_ratio = 1.5
disk_allocation_ratio = 9999.0
block_device_allocate_retries = 300
block_device_allocate_retries_interval = 10
block_device_creation_timeout = 300

We are going to deploy on multiple nodes so copy the example over

cp /usr/share/kolla-ansible/ansible/inventory/multinode /etc/kolla/

Change matching environment:

/etc/kolla/multinode

[control]
# These hostname must be resolvable from your deployment host
hp1
.
.
[network]
hp1
.
[compute]
hp2
.
[monitoring]
hp1
.
.
[storage]
hp1

Synology and Cinder

Yes that is right! Some Synology models have Openstack Cinder Volume plugin capabilities. I have created a Cinder conf file and I have added the nfs-1 backend, which is created by the nfs j2 file.

/etc/kolla/config/cinder.conf

[DEFAULT]
enabled_backends = iscsi-volume1,iscsi-volume2,nfs-1
default_volume_type = iscsi-volume1

[iscsi-volume1]
volume_driver = cinder.volume.drivers.synology.synology_iscsi.SynoISCSIDriver
volume_backend_name = iscsi-volume1
iscs_protocol = iscsi
iscsi_ip_address = 172.16.2.2
synology_admin_port = 5001
driver_use_ssl = True
synology_ssl_verify = False
synology_username = admin
synology_password = "PASSWORD"
synology_pool_name = volume1

[iscsi-volume2]
volume_driver = cinder.volume.drivers.synology.synology_iscsi.SynoISCSIDriver
volume_backend_name = iscsi-volume2
iscs_protocol = iscsi
iscsi_ip_address = 172.16.2.2
synology_admin_port = 5001
driver_use_ssl = True
synology_ssl_verify = False
synology_username = admin
synology_password = "PASSWORD"
synology_pool_name = volume2

Keystone LDAP domain

/etc/kolla/config/keystone/domains/keystone.helux.conf

[identity]
driver = ldap

[resource]
driver = sql

[assignment]
driver = sql

[role]
driver = sql

[ldap]
url = ldap://ldap.mngt.bh.helux.nl
suffix = dc=helux,dc=nl
query_scope = sub
user = cn=manager,dc=helux,dc=nl
password = 'PASSWORD'
use_dumb_member = False

user_tree_dn = ou=people,dc=helux,dc=nl
user_objectclass = inetOrgPerson

user_id_attribute = cn
user_name_attribute = cn
#user_mail_attribute = mail
#user_pass_attribute = userPassword
#user_enabled_attribute =

group_tree_dn = ou=group,dc=helux,dc=nl
group_objectclass = groupOfUniqueNames
group_id_attribute = cn
group_name_attribute = cn
group_member_attribute = uniqueMember
#group_desc_attribute = description

user_allow_create = false
user_allow_update = false
user_allow_delete = false
project_allow_create = false
project_allow_update = false
project_allow_delete = false
role_allow_create = false
role_allow_update = false
role_allow_delete = false
group_allow_create = false
group_allow_update = false
group_allow_delete = false

Deploy Docker containers

SSH keys

Kolla needs password-less root access to the nodes. As root user:

ssh-keygen
ssh-copy-id hp1
ssh-copy-id hp2

Pre-checks

kolla-ansible prechecks -i /etc/kolla/multinode

Deploy

Sometimes fluentd container check hangs… Ctrl-C the deploy command and do it again, no pre-checks

kolla-ansible deploy -i  /etc/kolla/multinode

When deployment is done you will see something like this:

PLAY RECAP *********************************************************************
hp1                        : ok=284  changed=134  unreachable=0    failed=0
hp2                        : ok=68   changed=26   unreachable=0    failed=0

We can go to post deploy now.

Cleanup

When deploying on other nodes or deploying again:

/usr/share/kolla-ansible/tools/cleanup-containers
/usr/share/kolla-ansible/tools/cleanup-host

Post deploy

kolla-ansible post-deploy
. /etc/kolla/admin-openrc.sh

Now Kolla comes with an initial setup script. We need to alter it a little bit to the network config we have.

/usr/share/kolla-ansible/init-runonce

.
.
IMAGE_URL=http://download.cirros-cloud.net/0.3.5/
IMAGE=cirros-0.3.5-x86_64-disk.img
IMAGE_NAME=cirros
EXT_NET_CIDR='172.16.3.0/24'
EXT_NET_RANGE='start=172.16.3.71,end=172.16.3.99'
EXT_NET_GATEWAY='172.16.3.1'
.
.
echo Configuring neutron.
openstack network create --external --provider-physical-network physnet1 \
    --provider-network-type flat public
openstack subnet create --no-dhcp \
    --allocation-pool ${EXT_NET_RANGE} --network public \
    --subnet-range ${EXT_NET_CIDR} --gateway ${EXT_NET_GATEWAY} public-subnet

openstack network create --provider-network-type vxlan demo-net
openstack subnet create --subnet-range 10.0.0.0/24 --network demo-net \
    --gateway 10.0.0.1 demo-subnet

openstack router create demo-router
openstack router add subnet demo-router demo-subnet
openstack router set --external-gateway public demo-router
.
.
if [ -r ~/.ssh/id_rsa.pub ]; then
    echo Configuring nova public key and quotas.
    openstack keypair create --public-key ~/.ssh/id_rsa.pub kolla
fi
.
.
# 40 instances
openstack quota set --instances 40 ${ADMIN_PROJECT_ID}

# 40 cores
openstack quota set --cores 40 ${ADMIN_PROJECT_ID}

# 32GB ram
openstack quota set --ram 32000 ${ADMIN_PROJECT_ID}

# add default flavors, if they don't already exist
if ! openstack flavor list | grep -q m1.tiny; then
    openstack flavor create --ram 512 --disk 1 --vcpus 1 m1.tiny
    openstack flavor create --ram 1024 --disk 4 --vcpus 1 m1.small
    openstack flavor create --ram 2048 --disk 8 --vcpus 2 m1.medium
    openstack flavor create --ram 4096 --disk 16 --vcpus 4 m1.large
fi

After it is altered to your needs, run it!

/usr/share/kolla-ansible/init-runonce

Get the admin password out of the passwords.yml

grep keystone_admin /etc/kolla/passwords.yml
keystone_admin_password: <STRING>

Use the <STRING> to login to Horizon Dashboard. In my case: http://cloud.mngt.bh.helux.nl

When stuff is running

Multiple storage backends

Run from Kolla deploy host:

cinder type-create nfs-1
cinder type-key nfs-1 set volume_backend_name=nfs-1
cinder type-create iscsi-volume1
cinder type-create iscsi-volume2
cinder type-key iscsi-volume1 set volume_backend_name=iscsi-volume1
cinder type-key iscsi-volume2 set volume_backend_name=iscsi-volume2
cinder type-list
cinder extra-specs-list

Bug in LDAP module of keystone

Run from controller host:

docker exec -ti -u0 keystone bash
cd /usr/lib/python2.7/site-packages/keystone/identity/backends/ldap
mv core.py core.py.org
curl https://git.openstack.org/cgit/openstack/keystone/plain/keystone/identity/backends/ldap/core.py?id=4e0029455ab45e3b9a15fe9fc151c14c502b7bdd -o core.py

Create domain