kvm setup for laptops with NetworkManager using bridges or openvswitch and NAT

On my workstation I have a static network setup: I don’t give an ip to eth0 but configure dhcp to give the ip to a bridge br0 attached to eth0. Then qemu-kvm creates tap devices attached to the bridge, getting ip’s in the same network as the host.

On my laptop I run NetworkManager, which does not play well with bridges. It seems that in other distributions you can tell the network configuration to use NetworkManager for certain interfaces only (which is still not exactly what I want). After lot of reading, I found a configuration that fits my needs.

The idea is to create a bridge, and instead of having it attached to eth0 (which is controlled by NetworkManager), we create the bridge in a separate network and use NAT to have the VMs access the internet.

I found a script by Amos Kong that setups the network (adapted to the paths of brctl in openSUSE):

#!/bin/bash

# Script used to add/remove setup of private bridge and dnsmasq
# @author Amos Kong 

brname='br0'

add_br()
{
    echo "add new private bridge"
    /sbin/brctl addbr $brname
    echo 1 > /proc/sys/net/ipv6/conf/$brname/disable_ipv6
    echo 1 > /proc/sys/net/ipv4/ip_forward
    /sbin/brctl stp $brname on
    /sbin/brctl setfd $brname 0
    ifconfig $brname 192.168.58.1
    ifconfig $brname up
    # add iptable entry as libvirt, then guest can access public network
    iptables -t nat -A POSTROUTING -s 192.168.58.254/24 ! -d 192.168.58.254/24 -j MASQUERADE
   /etc/init.d/dnsmasq stop
    /etc/init.d/tftpd-hpa stop 2>/dev/null
    dnsmasq --strict-order --bind-interfaces --listen-address 192.168.58.1 --dhcp-range 192.168.58.2,192.168.58.254 $tftp_cmd
}

del_br()
{
    echo "cleanup bridge setup"
    kill -9 `pgrep dnsmasq|tail -1`
    ifconfig $brname down
    /sbin/brctl delbr $brname
   iptables -t nat -D POSTROUTING -s 192.168.58.254/24 ! -d 192.168.58.254/24 -j MASQUERADE
}

# clean original setup first
del_br 2>/dev/null

if [[ $# > 0 ]];then
    if [[ $# = 2 ]];then
        # setup tftp function
       tftp_cmd=" --enable-tftp --tftp-root $1 --dhcp-boot $2 --dhcp-no-override"
    fi
    add_br
fi

Calling the script with no arguments will remove the bridge. Calling it with “1″ as argument setups the bridge and NAT and also runs dnsmasq (dhcp server and dns cache: zypper install dnsmasq) on the bridge. Calling it with 2 will also setup a tftp server on the dnsmasq process.

Then you need a pair of scripts for qemu, which are called with the tap device as a parameter.

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 up
/sbin/brctl addif ${switch} $1
/sbin/brctl setfd ${switch} 0
/sbin/brctl stp ${switch} off

And for bringing down the interface:

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 down
/sbin/brctl delif ${switch} $1

Then you run the VM like:

qemu-kvm -boot c -drive file=./disk.qcow2,if=virtio -m 2500 -net nic,macaddr=XX:XX:XX:XX:XX:XX -net tap,script=script-ifup,downscript=script-ifdown "$@"

You can use the same setup with openvswitch. There is a package in the network project of the build service, but the package is tied to the xenserver configuration so I did not get it running. I redid the package based on the Debian one, which not only is separated in subpackages but does not assume you are running Xen. The package is available here until the submit request is accepted.

Then change on the setup script the brctl addbr line to use ovs-vsctl:

add_br()
{
    echo "add new private bridge"
    ovs-vsctl add-br $brname
    echo 1 > /proc/sys/net/ipv6/conf/$brname/disable_ipv6
    echo 1 > /proc/sys/net/ipv4/ip_forward
    /sbin/brctl stp $brname on
    /sbin/brctl setfd $brname 0
    ...

And the for the qemu scripts, use ovs-vsctl add-port instead of brctl addif:

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 up
ovs-vsctl add-port ${switch} $1
/sbin/brctl setfd ${switch} 0
/sbin/brctl stp ${switch} off

And for bringing down the interface:

#!/bin/sh
switch='br0'
/sbin/ifconfig $1 0.0.0.0 down
ovs-vsctl del-port ${switch} $1