kvm_and_docker_on_one_host
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| kvm_and_docker_on_one_host [2019/07/26 13:47] – herwarth | kvm_and_docker_on_one_host [2020/04/12 09:10] (current) – [ZFS] herwarth | ||
|---|---|---|---|
| Line 11: | Line 11: | ||
| Choose Ubuntu minimal install and only select SSH server to be installed additionally. | Choose Ubuntu minimal install and only select SSH server to be installed additionally. | ||
| - | =====Configure | + | =====Configure===== |
| We are going to use netplan for network configuration. In our example we have one network interface with multiple VLANs on it. We create bridge interfaces using these VLAN interfaces. | We are going to use netplan for network configuration. In our example we have one network interface with multiple VLANs on it. We create bridge interfaces using these VLAN interfaces. | ||
| Line 46: | Line 46: | ||
| accept-ra: no | accept-ra: no | ||
| addresses: | addresses: | ||
| - | - x.x.x.x/24 | + | - x.x.2.x/24 |
| - | - x:x:x:x::x/64 | + | - x:x:x:2::x/64 |
| - | gateway4: x.x.x.1 | + | gateway4: x.x.2.1 |
| - | gateway6: x:x:x:x::1 | + | gateway6: x:x:x:2::1 |
| nameservers: | nameservers: | ||
| addresses: [ " | addresses: [ " | ||
| Line 72: | Line 72: | ||
| =====Remove unnecessary crap===== | =====Remove unnecessary crap===== | ||
| apt purge snapd | apt purge snapd | ||
| - | | + | |
| - | =====Install | + | ======KVM/libvirt====== |
| + | =====Installation===== | ||
| apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virt-manager | apt install qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils virt-manager | ||
| systemctl start libvirtd | systemctl start libvirtd | ||
| systemctl enable libvirtd | systemctl enable libvirtd | ||
| - | =====Configure | + | =====Configure===== |
| + | ====Network==== | ||
| Because we already have the bridges created with netplan, we only need to assign them to networking names in libvirt. | Because we already have the bridges created with netplan, we only need to assign them to networking names in libvirt. | ||
| <code - / | <code - / | ||
| Line 134: | Line 136: | ||
| | | ||
| </ | </ | ||
| - | =====Install | + | ====Systemd==== |
| + | Because we are going to use Docker | ||
| + | < | ||
| + | cd / | ||
| + | rm libvirt-bin.service | ||
| + | cp / | ||
| + | vi libvirtd.service | ||
| + | </ | ||
| + | <code - / | ||
| + | . | ||
| + | . | ||
| + | . | ||
| + | After=network.target | ||
| + | After=dbus.service | ||
| + | After=iscsid.service | ||
| + | After=apparmor.service | ||
| + | After=local-fs.target | ||
| + | After=remote-fs.target | ||
| + | After=systemd-machined.service | ||
| + | After=docker.service | ||
| + | Documentation=man: | ||
| + | Documentation=https:// | ||
| + | |||
| + | [Service] | ||
| + | . | ||
| + | . | ||
| + | . | ||
| + | </ | ||
| + | Now we need to reload systemd | ||
| + | systemctl daemon-reload | ||
| + | |||
| + | ======Docker====== | ||
| + | =====Installation===== | ||
| apt install docker.io | apt install docker.io | ||
| systemctl enable docker | systemctl enable docker | ||
| - | ====== | + | We need docker-compose and are going to install it via pip |
| + | apt install python3-pip | ||
| + | Now install docker-compose | ||
| + | pip3 install docker-compose | ||
| + | |||
| + | =====Configure===== | ||
| + | Docker is using iptables to mitigate traffic between containers. Because we are using VLANs with bridges, we do not want Docker to intervene network traffic. | ||
| + | <code - / | ||
| + | { | ||
| + | " | ||
| + | } | ||
| + | </ | ||
| + | When we stop docker to manipulate iptables it will not add the masquerade rule to the docker0 bridge. We need this iptables rule to make docker build work again. So first we are going to add the iptables rule manually. Find out which subnet is connected to docker0 | ||
| + | |||
| + | ip a s docker0 | ||
| + | < | ||
| + | 11: docker0: < | ||
| + | link/ether 02: | ||
| + | inet 172.17.0.1/ | ||
| + | | ||
| + | inet6 fe80:: | ||
| + | | ||
| + | </ | ||
| + | In this case the subnet is 172.17.0.0/ | ||
| + | iptables -t nat -A POSTROUTING -s 172.17.0.0/ | ||
| + | |||
| + | Now we have the rule added. Make it persistent. First install iptables-persistent | ||
| + | apt install iptables-persistent | ||
| + | It will tell you it created rules in: / | ||
| + | Check the rules.v4 file for the masquerade rule | ||
| + | |||
| + | cat / | ||
| + | < | ||
| + | # Generated by iptables-save v1.6.1 on Fri Jul 26 18:19:47 2019 | ||
| + | *nat | ||
| + | :PREROUTING ACCEPT [468: | ||
| + | :INPUT ACCEPT [22: | ||
| + | :OUTPUT ACCEPT [8:601] | ||
| + | : | ||
| + | -A POSTROUTING -s 172.17.0.0/ | ||
| + | COMMIT | ||
| + | # Completed on Fri Jul 26 18:19:47 2019 | ||
| + | # Generated by iptables-save v1.6.1 on Fri Jul 26 18:19:47 2019 | ||
| + | *filter | ||
| + | :INPUT ACCEPT [2328: | ||
| + | :FORWARD ACCEPT [3836: | ||
| + | :OUTPUT ACCEPT [1473: | ||
| + | : | ||
| + | -A FORWARD -j DOCKER-USER | ||
| + | -A DOCKER-USER -j RETURN | ||
| + | COMMIT | ||
| + | # Completed on Fri Jul 26 18:19:47 2019 | ||
| + | </ | ||
| + | ====ZFS==== | ||
| + | When you have a ZFS pool you want to use for storage | ||
| + | < | ||
| + | systemctl stop docker | ||
| + | rm -rf / | ||
| + | zfs create -o mountpoint=/ | ||
| + | </ | ||
| + | <code - / | ||
| + | { | ||
| + | " | ||
| + | " | ||
| + | } | ||
| + | </ | ||
| + | < | ||
| + | systemctl start docker | ||
| + | </ | ||
| + | Check with docker info | ||
| + | < | ||
| + | < | ||
| + | Client: | ||
| + | Debug Mode: false | ||
| + | |||
| + | Server: | ||
| + | | ||
| + | Running: 0 | ||
| + | Paused: 0 | ||
| + | Stopped: 0 | ||
| + | | ||
| + | | ||
| + | | ||
| + | Zpool: data | ||
| + | Zpool Health: ONLINE | ||
| + | Parent Dataset: data/ | ||
| + | Space Used By Parent: 98304 | ||
| + | Space Available: 965292744704 | ||
| + | Parent Quota: no | ||
| + | Compression: | ||
| + | . | ||
| + | . | ||
| + | . | ||
| + | </ | ||
| + | ====Networks==== | ||
| + | In our example we are not going to use containers in the routed subnet bridge called internet | ||
| + | < | ||
| + | docker network create -d macvlan --subnet=x.x.0.0/ | ||
| + | </ | ||
| + | < | ||
| + | docker network create -d macvlan --subnet=x.x.1.0/ | ||
| + | </ | ||
| + | < | ||
| + | docker network create -d macvlan --subnet=x.x.2.0/ | ||
| + | </ | ||
| + | Check it | ||
| + | docker network ls | ||
| + | < | ||
| + | NETWORK ID NAME DRIVER | ||
| + | 6ae8a5b99638 | ||
| + | 700edcbcc28b | ||
| + | 06d5b0706837 | ||
| + | cb04700d3481 | ||
| + | b0bdff0a6d53 | ||
| + | 320e38274915 | ||
| + | </ | ||
| + | |||
| + | {{tag> | ||
kvm_and_docker_on_one_host.1564148840.txt.gz · Last modified: by herwarth
Page Tools
