I have two NUCs with an M.2 disk and a SATA disk. I would like to make an Open Nebula cluster out of it keeping the SATA disks in sync using DRBD9 and ZFS. I want the Open Nebula front-end as a VM running on that cluster.

I created an Ubuntu 18.04 LTS VM using virt-manager on my Linux laptop. Make sure it has a minimum of 2GB memory. Start with an 8GB disk (makes transfer to NUC in the end faster) and expand when deploying on the NUC. Make sure OpenSSH is installed and running.

Open Nebula 5.10 is the latest as of writing. MariaDB 10.4 is the latest as of writing.

apt purge snapd ufw
apt install chrony
systemctl enable chrony

add-apt-repository ppa:linbit/linbit-drbd9-stack

wget -q -O- https://downloads.opennebula.org/repo/repo.key | apt-key add -
echo "deb https://downloads.opennebula.org/repo/5.10/Ubuntu/18.04 stable opennebula" | tee /etc/apt/sources.list.d/opennebula.list

apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
echo "deb [arch=arm64,amd64,ppc64el] http://mariadb.mirror.liquidtelecom.com/repo/10.4/ubuntu bionic main" | tee /etc/apt/sources.list.d/mariadb.list

apt update

apt -y install mariadb-server mariadb-client
mysql_secure_installation
mysql -u root -p
CREATE DATABASE opennebula;
GRANT ALL PRIVILEGES ON opennebula.* TO 'oneadmin' IDENTIFIED BY 'secretpassword';
FLUSH PRIVILEGES;

apt update
apt -y install opennebula opennebula-sunstone opennebula-gate opennebula-flow

!! DO NOT RUN THE SUGGESTED /usr/share/one/install_gems

/etc/one/oned.conf

- DB = [ BACKEND = "sqlite" ]
+ #DB = [ BACKEND = "sqlite" ]

+ DB = [ backend = "mysql",
+ server = "localhost",
+ port = 0,
+ user = "oneadmin",
+ passwd = "secretpassword",
+ db_name = "opennebula" ]

su - oneadmin
echo "oneadmin:userpassword" > ~/.one/one_auth

systemctl start opennebula opennebula-sunstone
systemctl enable opennebula opennebula-sunstone

Check if it is working

su - oneadmin -c "oneuser show"
USER 0 INFORMATION
ID              : 0
NAME            : oneadmin
GROUP           : oneadmin
PASSWORD        : *****
AUTH_DRIVER     : core
ENABLED         : Yes

TOKENS

USER TEMPLATE
TOKEN_PASSWORD="******"

VMS USAGE & QUOTAS

VMS USAGE & QUOTAS - RUNNING

DATASTORE USAGE & QUOTAS

NETWORK USAGE & QUOTAS

IMAGE USAGE & QUOTAS

When you see something like above, you can try an login to the web interface: http://<YOUR_IP_HERE>:9869

apt update
apt install linstor-controller linstor-client

systemctl enable linstor-controller
systemctl start linstor-controller

For now we leave the controller as is. We need to install the back-ends first.

Make sure you follow this chapter on every back-end node you want to use. This is going to be a physical installation on the NUCs in my case. Again I am using Ubuntu 18.04 because of the availability of a DRBD9 repository and ZFS! Make sure you do a minimal Ubuntu install with just OpenSSH.

apt purge snapd ufw
apt install chrony
systemctl enable chrony

add-apt-repository ppa:linbit/linbit-drbd9-stack

wget -q -O- https://downloads.opennebula.org/repo/repo.key | apt-key add -
echo "deb https://downloads.opennebula.org/repo/5.10/Ubuntu/18.04 stable opennebula" | tee /etc/apt/sources.list.d/opennebula.list

apt update

I am using a USB-C gigabit ethernet controller and udev creates a network device name with the MAC address of the device. This is not what I want. I just want it to have a generic name. In this case I have chosen eth0.

cp /lib/udev/rules.d/73-usb-net-by-mac.rules /etc/udev/rules.d

Edit the copied file

/etc/udev/rules.d73-usb-net-by-mac.rules

-    IMPORT{builtin}="net_id", NAME="$env{ID_NET_NAME_MAC}"
+    IMPORT{builtin}="net_id", NAME="eth0"

apt install opennebula-node
apt install qemu-utils virt-manager
apt install zfsutils-linux

systemctl restart libvirtd
systemctl restart libvirt-bin
systemctl enable libvirtd

In my case the sata disk is /dev/sda. I am not going to create a partition table on it.

zpool create -f data /dev/sda

I do not want to mount the ZFS filesystem. Just using it for zvols with DRBD9.

/etc/default/zfs

- ZFS_MOUNT='yes'
+ ZFS_MOUNT='no'
- ZFS_UNMOUNT='yes'
+ ZFS_UNMOUNT='no'

We need to do a reboot for the network interface name change

shutdown -r now

I want to use a bridge interface for management. Makes it easy in Open Nebula to configure bridge as VM network interface. Use as you like

/etc/netplan/01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      accept-ra: no
      dhcp4: no
      dhcp6: no
    eth0:
      accept-ra: no
      dhcp4: no
      dhcp6: no
  bridges:
    mgmt:
      accept-ra: no
      addresses:
        - 172.16.2.x/24
        - 2001:x:x:x::x/64
      gateway4: 172.16.2.254
      gateway6: 2001:x:x:x::254
      nameservers:
        addresses: [ "172.16.2.y", "208.67.222.222" ]
        search: [ mgmt.heitmann.nl ]
      interfaces: [eno1]