I would like to have a single server which serves as Docker host and KVM at the same time using the same network bridges. I am not going to use NAT in Docker and will use the same L2 bridges used by KVM. I am going to use Ubuntu 18.04 LTS

• Download Ubuntu 18.04 LTS from http://www.ubuntu.com

Choose Ubuntu minimal install and only select SSH server to be installed additionally.

We are going to use netplan for network configuration. In our example we have one network interface with multiple VLANs on it. We create bridge interfaces using these VLAN interfaces.

/etc/netplan/01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      accept-ra: no
      dhcp4: no
      dhcp6: no
  vlans:
    vlan8:
      id: 8
      link: eno1
      accept-ra: no
      dhcp4: no
      dhcp6: no
    vlan9:
      id: 9
      link: eno1
      accept-ra: no
      dhcp4: no
      dhcp6: no
    vlan10:
      id: 10
      link: eno1
      accept-ra: no
      dhcp4: no
      dhcp6: no
  bridges:
    mgmt:
      accept-ra: no
      addresses:
        - x.x.x.x/24
        - x:x:x:x::x/64
      gateway4: x.x.x.1
      gateway6: x:x:x:x::1
      nameservers:
        addresses: [ "x.x.x.x", "208.67.222.222" ]
        search: [ mgmt ]
      interfaces: [eno1]
    internet:
      accept-ra: no
      dhcp4: no
      dhcp6: no
      interfaces: [vlan8]
    dmz:
      accept-ra: no
      dhcp4: no
      dhcp6: no
      interfaces: [vlan9]
    local:
      accept-ra: no
      dhcp4: no
      dhcp6: no
      interfaces: [vlan10]

apt purge snapd

apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virt-manager
systemctl start libvirtd
systemctl enable libvirtd

Because we already have the bridges created with netplan, we only need to assign them to networking names in libvirt.

/etc/libvirt/qemu/networks/internet.xml

<network>
  <name>internet</name>
  <forward mode='bridge'/>
  <bridge name='internet'/>
</network>

/etc/libvirt/qemu/networks/dmz.xml

<network>
  <name>dmz</name>
  <forward mode='bridge'/>
  <bridge name='dmz'/>
</network>

/etc/libvirt/qemu/networks/local.xml

<network>
  <name>local</name>
  <forward mode='bridge'/>
  <bridge name='local'/>
</network>

/etc/libvirt/qemu/networks/mgmt.xml

<network>
  <name>mgmt</name>
  <forward mode='bridge'/>
  <bridge name='mgmt'/>
</network>

Now we have to activate the xmls

virsh net-destroy default
virsh net-undefine default
virsh net-define internet.xml
virsh net-define dmz.xml
virsh net-define local.xml
virsh net-define mgmt.xml
virsh net-start internet
virsh net-start dmz
virsh net-start local
virsh net-start mgmt
virsh net-autostart internet
virsh net-autostart dmz
virsh net-autostart local
virsh net-autostart mgmt

Check if everything is ok

virsh net-list

 Name                 State      Autostart     Persistent
----------------------------------------------------------
 dmz                  active     yes           yes
 internet             active     yes           yes
 local                active     yes           yes
 mgmt                 active     yes           yes

apt install docker.io
systemctl enable docker

Docker is using iptables to mitigate traffic between containers. Because we are using VLANs with bridges, we do not want Docker to intervene network traffic.

/etc/docker/daemon.json

{
"iptables": false
}

systemctl restart docker

In our example we are not going to use containers in the routed subnet bridge called internet

docker network create -d macvlan --subnet=x.x.0.0/24 --gateway=x.x.0.1 --ip-range=x.x.0.128/28 -o parent=local local

docker network create -d macvlan --subnet=x.x.1.0/24 --gateway=x.x.1.1 --ip-range=x.x.1.128/28 --subnet=x:x:x:1::/64 --gateway=x:x:x:1::1 --ip-range=x:x:x:1::128/124 -o parent=dmz --ipv6 dmz

docker network create -d macvlan --subnet=x.x.2.0/24 --gateway=x.x.2.1 --ip-range=x.x.2.128/28 --subnet=x:x:x:2::/64 --gateway=x:x:x:2::1 --ip-range=x:x:x:2::128/124 -o parent=mgmt --ipv6 mgmt