Setup hyperd with flannel network

Flannel

Flannel is a virtual network that gives a subnet to each host for use with container runtimes.

Platforms like Google’s Kubernetes assume that each container (pod) has a unique, routable IP inside the cluster. The advantage of this model is that it reduces the complexity of doing port mapping.

flannel runs an agent, flanneld, on each host and is responsible for allocating a subnet lease out of a preconfigured address space. flannel uses etcd to store the network configuration, allocated subnets, and auxiliary data (such as host’s IP). The forwarding of packets is achieved using one of several strategies that are known as backends. The simplest backend is udp and uses a TUN device to encapsulate every IP fragment in a UDP packet, forming an overlay network. The following diagram demonstrates the path a packet takes as it traverses the overlay network:

Flannel install

First install etcd:

curl -L https://github.com/coreos/etcd/releases/download/v3.0.3/etcd-v3.0.3-linux-amd64.tar.gz -o etcd-v3.0.3-linux-amd64.tar.gz
tar xzvf etcd-v3.0.3-linux-amd64.tar.gz
cp etcd-v3.0.3-linux-amd64/{etcd,etcdctl} /usr/bin
rm -rf etcd-v3.0.3-linux-amd64 etcd-v3.0.3-linux-amd64.tar.gz

Then, install flannel:

curl -L https://github.com/coreos/flannel/releases/download/v0.5.5/flannel-0.5.5-linux-amd64.tar.gz -o flannel-0.5.5-linux-amd64.tar.gz
tar zxvf flannel-0.5.5-linux-amd64.tar.gz
cp flannel-0.5.5/flanneld /usr/bin
rm -rf flannel-0.5.5*

Start etcd and setup default network:

nohup etcd --advertise-client-urls 'http://192.168.33.10:2379' --listen-client-urls 'http://192.168.33.10:2379' &
etcdctl --endpoints=192.168.33.10:2379 set /coreos.com/network/config  '{ "Network": "172.168.0.0/16", "Backend": { "Type": "vxlan", "VNI": 2000 } }'

Start flanneld on all nodes:

nohup flanneld -etcd-endpoints=http://192.168.33.10:2379 -iface=eth1 &

Hyperd install

apt-get install qemu-system-x86 -y
curl -sSL http://hypercontainer.io/install | bash

Configure hyperd to use subnet provided by flannel:

source /run/flannel/subnet.env
brctl addbr docker0
ip addr add dev docker0 ${FLANNEL_SUBNET}
ip link set docker0 up

cat >/etc/hyper/config <<EOF
Kernel=/var/lib/hyper/kernel
Initrd=/var/lib/hyper/hyper-initrd.img
Hypervisor=qemu
StorageDriver=devicemapper
Bridge=docker0
BridgeIP=${FLANNEL_SUBNET}
EOF

nohup hyperd --nondaemon --v=3 &

Test

[email protected]:~# hyper run -d busybox
POD id is pod-hZviZLulsb
Time to run a POD is 3648 ms
[email protected]:~# hyper exec pod-hZviZLulsb ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 52:54:51:e5:db:2f brd ff:ff:ff:ff:ff:ff
    inet 172.168.12.3/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:51ff:fee5:db2f/64 scope link
       valid_lft forever preferred_lft forever

[email protected]:~# hyper run -d busybox
POD id is pod-GbccOdYKjK
Time to run a POD is 3631 ms
[email protected]:~# hyper exec pod-GbccOdYKjK ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 52:54:da:0c:b6:cd brd ff:ff:ff:ff:ff:ff
    inet 172.168.95.3/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:daff:fe0c:b6cd/64 scope link
       valid_lft forever preferred_lft forever
[email protected]:~# hyper exec pod-GbccOdYKjK ping -c3 172.168.12.3
PING 172.168.12.3 (172.168.12.3): 56 data bytes
64 bytes from 172.168.12.3: seq=0 ttl=62 time=57.400 ms
64 bytes from 172.168.12.3: seq=1 ttl=62 time=6.563 ms
64 bytes from 172.168.12.3: seq=2 ttl=62 time=1.580 ms

--- 172.168.12.3 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 1.580/21.847/57.400 ms

Reference

Comments

comments powered by Disqus