libvirt Networking Handbook¶

This guide demonstrates the most common aspects of libvirt networking, whether running virtual machines (VMs) on a dedicated server or within a home lab.

How to choose a network type

On a dedicated server — where VMs often need to be publicly accessible — a Bridged network is ideal and allows each VM to bind to its own public IPv4 and IPv6 addresses. If bridging is not possible, create a Routed network. If the server has limited public IPv4 addresses, a NAT-based network that forwards incoming connections may be the only option.

Inside an intranet or home lab, a NAT-based network gives VMs outbound network access. If VMs are running services that must be accessible from other systems on the LAN, create a Bridged network (for an Ethernet connected libvirt host) or a Routed network (for a wirelessly connected libvirt host).

If you want to prevent libvirt from automatically inserting iptables rules, create a Bridged network, Custom routed network, or Custom NAT-based network.

Bridged network¶

A bridged network shares a real Ethernet device with virtual machines (VMs). Each VM can bind directly to any available IPv4 or IPv6 addresses on the LAN, just like a physical computer. Bridging offers the best performance and the least headache out of the libvirt network types.

Limitations¶

The libvirt server must be connected to the LAN via Ethernet. If it is connected wirelessly, a Routed network or NAT-based network are the only options.

Limitations for dedicated servers

A bridge is only possible when there are enough IP addresses to allocate one per VM. This is not a problem for IPv6, as hosting providers usually provide many free IPv6 addresses. However, extra IPv4 addresses are rarely free. If you only have one public IPv4 address (and need to serve clients over IPv4), either buy more IPv4 addresses or create a NAT-based network.

Hosting providers often only allow the MAC address of the server to bind to IP addresses on the LAN, which prevents bridging. Your provider may let you rent a private VLAN that allows VMs to bind directly to IP addresses, but if this is too costly then consider a Routed network.

Initial steps¶

In this example:

The server has an Ethernet device called eth0.
VMs share eth0, which is enslaved into a bridge called br0.
The hosting provider has allocated two address blocks (see CIDR notation):
- one public IPv4 address block (203.0.113.160/29)
- one public IPv6 address block (2001:db8::/64)
The server binds statically to 203.0.113.166 and 2001:db8::1.
VMs can bind to any available IPv4 and IPv6 addresses in the allocated blocks.

Identify the MAC address of the Ethernet device for later.

# ip address show dev eth0 | awk '$1=="link/ether" {print $2}'
19:7c:3b:92:ec:ee

For performance and security reasons, disable netfilter for bridges. Create /etc/sysctl.d/bridge.conf with these contents:

net.bridge.bridge-nf-call-ip6tables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-call-arptables=0

Create /etc/udev/rules.d/99-bridge.rules with the following contents. This udev rule applies the sysctl settings above when the bridge module is loaded. (If using Linux kernel 3.18 or later, change KERNEL=="bridge" to KERNEL=="br_netfilter".)

ACTION=="add", SUBSYSTEM=="module", KERNEL=="bridge", RUN+="/sbin/sysctl -p /etc/sysctl.d/bridge.conf"

Intranet or home lab¶

The example is for a dedicated server, but the instructions are equally useful for an Intranet or home lab. There are two small differences:

Devices on the LAN are likely allocated private IP addresses (eg, 192.168.0.0/24) by a local router. Simply replace the public addresses in the example with appropriate private addresses for your LAN.
You may need to configure the router to always assign the same IP address to the MAC address of the server (and to any VMs that need a static address).

Configure the bridge¶

Note

NetworkManager has supported bridges since version 0.9.8, so there is usually no need to disable it. Even the ancient version of NetworkManager on Red Hat Enterprise Linux 6 is heavily patched and supports bridges.

Follow the instructions below for either Red Hat Enterprise Linux or Debian.

Red Hat Enterprise Linux (or Fedora)¶

Install the bridge-utils software.

# yum install bridge-utils

/etc/sysconfig/ifcfg-eth0 should already exist. Save a copy of the original file somewhere safe, then replace the contents with the following:

DEVICE=eth0
NAME=eth0
# Use the MAC address identified above.
HWADDR=19:7c:3b:92:ec:ee
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0

Create /etc/sysconfig/ifcfg-br0 with these contents:

# If unsure what NETMASK, GATEWAY or IPV6_DEFAULTGW should be, check the
# original copy of ifcfg-eth0 or ask your hosting provider.

DEVICE=br0
NAME=br0
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
# If you want to turn on Spanning Tree Protocol, ask your hosting
# provider first as it may conflict with their network.
STP=off
# If STP is off, set to 0. If STP is on, set to 2 (or greater).
DELAY=0

IPADDR=203.0.113.166
NETMASK=255.255.255.248
GATEWAY=203.0.113.161

IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6ADDR=2001:db8::1
# The gateway interface (ie, '%br0') must match the name of the bridge.
IPV6_DEFAULTGW=fe80::1%br0

If using NetworkManager:

# nmcli connection reload
# nmcli connection down eth0 && nmcli connection up eth0

If using NetworkManager on Red Hat Enterprise Linux 6:

# echo "NM_BOND_BRIDGE_VLAN_ENABLED=yes" >> /etc/sysconfig/network
# service NetworkManager restart

If not using NetworkManager:

# ifdown eth0 && ifup eth0 && ifup br0

Debian¶

Install the bridge-utils software.

# apt-get install bridge-utils

Add the following text to /etc/network/interfaces:

# If unsure what 'netmask' or 'gateway' should be, ask your hosting provider.

iface eth0 inet manual

auto br0
iface br0 inet static
    # Use the MAC address identified above.
    hwaddress ether 19:7c:3b:92:ec:ee
    address 203.0.113.166
    netmask 255.255.255.248
    gateway 203.0.113.161

    bridge_ports eth0
    # If you want to turn on Spanning Tree Protocol, ask your hosting
    # provider first as it may conflict with their network.
    bridge_stp off
    # If STP is off, set to 0. If STP is on, set to 2 (or greater).
    bridge_fd 0

iface br0 inet6 static
    address 2001:db8::1
    netmask 64
    gateway fe80::1
    autoconf 0

Bring the bridge up.

# ip address flush eth0 scope global && ifup br0

Check the bridge¶

Check that the bridge is up and the Ethernet device is listed as an interface. If the steps above did not work or you have a complicated network environment, you may need to reboot the server.

# brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.197c3b92ecee       no              eth0

Configure virtual machines¶

New VM¶

# virt-install --network bridge=br0 ...

Optionally, pass --network more than once to create additional virtual Ethernet interfaces for the VM.

# virt-install --network bridge=br0 --network network=default ...

Existing VM¶

Open the XML configuration for the VM in a text editor.

# virsh edit name-of-vm

There should already be an <interface> section that configures a virtual Ethernet interface for the VM. Note down the MAC address.

<interface type="network">
   <source network="default"/>
   <mac address="52:54:00:4f:47:f2"/>
</interface>

To reconfigure the virtual Ethernet device, replace the <interface> section with the following contents. Use the MAC address noted above, otherwise the MAC address of the VM will change.

<interface type="bridge">
  <source bridge="br0"/>
  <mac address="52:54:00:4f:47:f2"/>
</interface>

To add an additional Ethernet interface, append a new <interface> section. libvirt generates a random MAC for the new interface if <mac> is omitted.

<interface type="bridge">
   <source bridge="br0"/>
</interface>

Reboot the VM to apply the changes. You may need to amend the VM’s network initialization scripts to account for the network interface changes.

Routed network¶

A routed network is usually only used when a Bridged network is unavailable, either due to hosting provider restrictions or because the libvirt server is connected wirelessly to the LAN. Virtual machines (VMs) have their own IP addresses, but do not bind directly to them. Instead, packets destined for those addresses are statically routed to the libvirt server and forwarded to VMs (without using NAT).

Limitations¶

Unfortunately, libvirt’s built-in routed network automatically inserts iptables rules whether you want them or not, in an order that is difficult to control. If you would rather be in full control and prevent libvirt from interfering, create a Custom routed network instead.

On a dedicated server, a routed network is only possible when there are enough IP addresses to allocate one per VM. This is not a problem for IPv6, as hosting providers usually provide many free IPv6 addresses. However, extra IPv4 addresses are rarely free. If you only have one public IPv4 address (and need to serve clients over IPv4), either buy more IPv4 addresses or create a NAT-based network.

Initial steps¶

In this example:

The server has an Ethernet device called eth0.
VMs bind to a virtual bridge called virbr1, which is created by libvirt.
The hosting provider has allocated two address blocks (see CIDR notation):
- one public IPv4 address block (203.0.113.80/28)
- one public IPv6 address block (2001:db8::/56)
The server binds statically to 203.0.113.86 and 2001:db8::1.

To begin, choose which IP addresses to make available to VMs. In this example, a subnet of six IPv4 addresses (203.0.113.88/29) will be sliced from the allocated address block and made available to VMs. In an Intranet or home lab, you might choose a whole /24 subnet (eg, 192.168.50.0/24).

For IPv6, always use a /64 subnet prefix to avoid breaking various IPv6 features. In this example, 2001:db8::/56 can be sliced into 256 different /64 subnets, one of which (2001:db8:aa::/64) will be made available to VMs. (Most hosting providers allocate a /64 block for every server, and should allocate a free /56 on request.)

Configure static routes¶

The LAN router is unaware that the subnets chosen above are located on the libvirt server, so network packets can only reach VMs if you configure static routes on the LAN router. In this example, two static routes are required:

ip -4 route add 203.0.113.88/29 via 203.0.113.86
ip -6 route add 2001:db8:aa::/64 via 2001:db8::1

For a dedicated server, ask your hosting provider to configure these routes for you. For an Intranet or home lab, consult the documentation for your router.

Note

If the router in your home lab is consumer-grade, it might have buggy support for static routes or it might not allow static routes at all. If possible, install alternative firmware such as OpenWRT or Merlin. (A painful last resort is to add static routes to every client on the LAN.)

Configure the virtual network¶

Create /tmp/mynetwork1.xml with the following contents. If only a single protocol version is required, omit either the IPv4 or IPv6 <ip> block. (Optionally, use DHCP host entries to always assign the same IP address to a particular VM.)

<network>
  <name>mynetwork1</name>
  <bridge name="virbr1" />
  <forward mode="route"/>
  <ip address="203.0.113.88" netmask="255.255.255.248">
    <dhcp>
      <range start="203.0.113.89" end="203.0.113.94"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="2001:db8:aa::1" prefix="64"/>
</network>

IPv6 addresses are allocated via stateless address autoconfiguration (SLAAC). Alternatively, you may want to allocate IPv6 addresses via DHCPv6 instead, in which case add a <dhcp> element with a range of addresses to offer to VMs:

<network>
  <name>mynetwork1</name>
  <bridge name="virbr1" />
  <forward mode="route"/>
  <ip address="203.0.113.88" netmask="255.255.255.248">
    <dhcp>
      <range start="203.0.113.89" end="203.0.113.94"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="2001:db8:aa::1" prefix="64">
    <dhcp>
      <range start="2001:db8:aa::1000" end="2001:db8:aa::1fff"/>
    </dhcp>
  </ip>
</network>

Use /tmp/mynetwork1.xml to define a new network.

# virsh net-define /tmp/mynetwork1.xml
# virsh net-autostart mynetwork1
# virsh net-start mynetwork1

Multiple virtual networks

You can create as many routed networks as required. Simply choose a different name for the network (eg, mynetwork2), a different name for the virtual bridge (eg, virbr2), and a different range of IP addresses. Also see Multiple networks.

Configure virtual machines¶

New VM¶

# virt-install --network network=mynetwork1 ...

Optionally, pass --network more than once to create additional virtual Ethernet interfaces for the VM.

# virt-install --network network=mynetwork1 --network network=mynetwork2 ...

Existing VM¶

Open the XML configuration for the VM in a text editor.

# virsh edit name-of-vm

There should already be an <interface> section that configures a virtual Ethernet interface for the VM. Note down the MAC address.

<interface type="network">
   <source network="default"/>
   <mac address="52:54:00:4f:47:f2"/>
</interface>

To reconfigure the virtual Ethernet interface, replace the <interface> section with the following contents. Use the MAC address noted above, otherwise the MAC address of the VM will change.

<interface type="network">
   <source network="mynetwork1"/>
   <mac address="52:54:00:4f:47:f2"/>
</interface>

To add an additional virtual Ethernet interface, append a new <interface> section. libvirt generates a random MAC for the new interface if <mac> is omitted.

<interface type="network">
   <source network="mynetwork1"/>
</interface>

Reboot the VM to apply the changes. You may need to amend the VM’s network initialization scripts to account for the network interface changes.

NAT-based network¶

A NAT-based network is ideal if virtual machines (VMs) only need outbound IPv4 network access. The libvirt server acts as a router, and VM traffic appears to originate from the IPv4 address of the server.

Limitations¶

The default virtual network is NAT-based (with a fragile hook system to forward incoming connections). Unfortunately, it automatically inserts iptables rules whether you want them or not — in an order that is difficult to control — unless you disable the default network completely.

If you would rather be in full control and prevent libvirt from interfering, create a Custom NAT-based network instead.

Configure the default network¶

The default NAT-based network should already be available after installing libvirt.

# virsh net-info default
Name:           default
UUID:           f63c210f-67e7-4aed-8d21-066d9fd6c7d6
Active:         no
Persistent:     yes
Autostart:      no
Bridge:         virbr0

# virsh net-autostart default
# virsh net-start default

If the default network is missing, create /tmp/default.xml with the following contents. (Optionally, use DHCP host entries to always assign the same IP address to a particular VM.)

<network>
  <name>default</name>
  <bridge name="virbr0"/>
  <forward mode="nat"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
</network>

Use /tmp/default.xml to create the default network.

# virsh net-define /tmp/default.xml
# virsh net-start default
# virsh net-autostart default

Multiple virtual networks¶

You can create as many NAT-based networks as required. Simply choose a different name for the network (eg, default2), a different name for the virtual bridge (eg, virbr2), and a different range of IP addresses. Also see Multiple networks.

Forward incoming connections¶

Note

This step is optional. It is only necessary if one or more VMs are running services (eg, web applications) that need to be available over the network.

If one of the VMs has a web application listening on ports 80/443, connections to those ports on the server can be forwarded to the VM using a fragile hook system.

The main limitation is that a specific port on the server can only be forwarded to a single VM. This is problematic if many VMs are fighting over ports 80/443. One option is to forward connections to ports 80/443 on the server to a VM running a reverse proxy (eg, NGINX or HAProxy), which can then proxy those connections to other VMs. Alternatively, run a reverse proxy on the libvirt server itself.

If you want more control over your firewall than the hook system can provide, create a Custom NAT-based network.

Configure IPv6¶

Note

This step is optional.

NAT makes little sense for IPv6, so create a Routed network instead. A routed network can be run alongside a NAT-based network if both are needed.

If preferred, you can avoid creating a separate routed network and allow VMs on a NAT-based network to bind to IPv6 addresses. However, no NAT is performed for IPv6 traffic so you still have to configure static routes (see Routed network). Open the XML configuration for the default network in a text editor.

# virsh net-edit default

Replace the configuration with the following content. Addresses are allocated via stateless address autoconfiguration (SLAAC).

<network>
  <name>default</name>
  <bridge name="virbr0"/>
  <forward mode="nat"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="2001:db8::1" prefix="64"/>
</network>

Alternatively, you may want to allocate addresses via DHCPv6 instead of SLAAC. Add a <dhcp> element with a range of addresses to offer to VMs. (Optionally, use DHCP host entries to always assign the same IP address to a particular VM.)

<network>
  <name>default</name>
  <bridge name="virbr0"/>
  <forward mode="nat"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="2001:db8::1" prefix="64">
    <dhcp>
      <range start="2001:db8::1000" end="2001:db8::1fff"/>
    </dhcp>
  </ip>
</network>

Configure virtual machines¶

New VM¶

# virt-install --network network=default ...

Optionally, pass --network more than once to create additional virtual Ethernet interfaces for the VM.

# virt-install --network network=default --network network=default2 ...

Existing VM¶

Open the XML configuration for the VM in a text editor.

# virsh edit name-of-vm

To configure a virtual Ethernet interface for the VM, add an <interface> section. If required, you can add multiple <interface> sections.

<interface type="network">
   <source network="default"/>
</interface>

Reboot the VM to apply the changes. You may need to amend the VM’s network initialization scripts to account for the network interface changes.

Custom routed network¶

libvirt’s built-in Routed network has some limitations. Follow the steps below to overcome these limitations and take control of your server environment. There are four main components: a dummy network interface, a virtual bridge, some iptables rules, and dnsmasq.

Note

If you are uncomfortable with iptables, you might prefer to stick with the built-in Routed network.

Initial steps¶

In this example:

The server has an Ethernet device called eth0.
VMs bind to a virtual bridge called virbr10.
The hosting provider has allocated two address blocks (see CIDR notation):
- one public IPv4 address block (203.0.113.80/28)
- one public IPv6 address block (2001:db8::/56)
The server binds statically to 203.0.113.86 and 2001:db8::1.

Configure static routes¶

The LAN router is not yet aware that packets destined for the subnets above need to be routed to the libvirt server. Packets will only reach VMs if you configure static routes on the LAN router. In this example, two static routes are required:

ip -4 route add 203.0.113.88/29 via 203.0.113.86
ip -6 route add 2001:db8:aa::/64 via 2001:db8::1

For a dedicated server, ask your hosting provider to configure these routes for you. For an Intranet or home lab, consult the documentation for your router.

Note

Create a dummy interface¶

A bridge inherits the MAC address of the first interface that is attached, so it will keep changing unless the same VM is always powered on first. To keep the MAC address constant, create a dummy network interface with a chosen MAC address and attach it to the bridge before anything else.

Choose a MAC address for the virtual bridge. Use hexdump to generate a random MAC address in the format that libvirt expects (52:54:00:xx:xx:xx for KVM, 00:16:3e:xx:xx:xx for Xen).

# hexdump -vn3 -e '/3 "52:54:00"' -e '/1 ":%02x"' -e '"\n"' /dev/urandom
52:54:00:7e:27:af

Create a dummy network interface called virbr10-dummy and set the MAC address to the one generated above.

# ip link add virbr10-dummy address 52:54:00:7e:27:af type dummy

To create a persistent dummy interface at every boot, follow the instructions for Red Hat Enterprise Linux, Fedora, or Debian.

Create a virtual bridge¶

A Linux bridge without a real Ethernet device is considered virtual. To manually create the virtual bridge, run these commands:

# brctl addbr virbr10
# brctl stp virbr10 on
# brctl addif virbr10 virbr10-dummy
# ip address add 203.0.113.88/29 dev virbr10 broadcast 203.0.113.95
# ip address add 2001:db8:aa::/64 dev virbr10

Rather than creating the bridge manually, follow the instructions for Red Hat Enterprise Linux, Fedora, or Debian (so that the bridge is created at every boot).

Configure the firewall¶

Note

You might want to disable the default network before you continue.

When network packets destined for VMs (203.0.113.88/29) arrive at the libvirt server, they are forwarded to the virtual bridge. Modify some kernel parameters to allow this.

# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
# echo "net.ipv4.conf.all.forwarding=1" >> /etc/sysctl.conf
# echo "net.ipv6.conf.all.forwarding=1" >> /etc/sysctl.conf
# sysctl -p

Allow IPv4 forwarding for iptables.

# This format is understood by iptables-restore. See `man iptables-restore`.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]

... snipped ...
# Allow inbound traffic to the private subnet.
-A FORWARD -d 203.0.113.88/29 -o virbr10 -j ACCEPT
# Allow outbound traffic from the private subnet.
-A FORWARD -s 203.0.113.88/29 -i virbr10 -j ACCEPT
# Allow traffic between virtual machines.
-A FORWARD -i virbr10 -o virbr10 -j ACCEPT
# Reject everything else.
-A FORWARD -i virbr10 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o virbr10 -j REJECT --reject-with icmp-port-unreachable
... snipped ...
COMMIT

Allow IPv6 forwarding for ip6tables.

# This format is understood by ip6tables-restore. See `man ip6tables-restore`.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]

... snipped ...
# Allow inbound traffic to the private subnet.
-A FORWARD -d 2001:db8:aa::/64 -o virbr10 -j ACCEPT
# Allow outbound traffic from the private subnet.
-A FORWARD -s 2001:db8:aa::/64 -i virbr10 -j ACCEPT
# Allow traffic between virtual machines.
-A FORWARD -i virbr10 -o virbr10 -j ACCEPT
# Reject everything else.
-A FORWARD -i virbr10 -j REJECT --reject-with icmp6-port-unreachable
-A FORWARD -o virbr10 -j REJECT --reject-with icmp6-port-unreachable
... snipped ...
COMMIT

Run dnsmasq¶

Note

This step is optional, as VMs can bind to addresses without DHCP and can use their own DNS resolver. However, it is recommended unless you know what you are doing.

On the libvirt server, it is common to run a DHCP server (to decide which IP address to lease to each VM) and a DNS server (to respond to queries from VMs). dnsmasq is ideal for both of these tasks.

Create files and directories needed for dnsmasq.

# mkdir -p /var/lib/dnsmasq/virbr10
# touch /var/lib/dnsmasq/virbr10/hostsfile
# touch /var/lib/dnsmasq/virbr10/leases

Create /var/lib/dnsmasq/virbr10/dnsmasq.conf with these contents:

# Only bind to the virtual bridge. This avoids conflicts with other running
# dnsmasq instances.
except-interface=lo
bind-dynamic
interface=virbr10

# If using dnsmasq 2.62 or older, remove "bind-dynamic" and "interface" lines
# and uncomment these lines instead:
#bind-interfaces
#listen-address=203.0.113.88
#listen-address=2001:db8:aa::1

# IPv4 addresses to offer to VMs. This should match the chosen subnet.
dhcp-range=203.0.113.89,203.0.113.94

# Set this to at least the total number of addresses in DHCP-enabled subnets.
dhcp-lease-max=1000

# Assign IPv6 addresses via stateless address autoconfiguration (SLAAC).
dhcp-range=2001:db8:aa::,ra-only

# Assign IPv6 addresses via DHCPv6 instead (requires dnsmasq 2.64 or later).
# Remember to allow all incoming UDP port 546 traffic on the VM.
#dhcp-range=2001:db8:aa::1000,2001:db8:aa::1fff
#enable-ra
#dhcp-lease-max=5000

# File to write DHCP lease information to.
dhcp-leasefile=/var/lib/dnsmasq/virbr10/leases
# File to read DHCP host information from.
dhcp-hostsfile=/var/lib/dnsmasq/virbr10/hostsfile
# Avoid problems with old or broken clients.
dhcp-no-override
# https://www.redhat.com/archives/libvir-list/2010-March/msg00038.html
strict-order

Note

dnsmasq only supports SLAAC in version 2.64 onwards. If using an older version, see Run radvd.

Optionally, use hostsfile to always assign the same IPv4 address to a particular VM using its MAC address. dnsmasq only reads changes after receiving a SIGHUP.

# echo "52:54:00:3b:9a:2b,203.0.113.90" \
      >> /var/lib/dnsmasq/virbr10/hostsfile
# killall -SIGHUP -e dnsmasq

Optionally, use hostsfile to always assign the same IPv6 address to a particular VM using its DHCP Unique Identifier (DUID). For example, if a VM is currently bound to 2001:db8:aa::1010, determine the DUID of the VM using the leases file and then add a hostsfile entry.

# awk '$3=="2001:db8:aa::1010" {print $NF}' /var/lib/dnsmasq/virbr10/leases
00:01:00:01:1a:ff:2b:ff:52:54:00:3b:9a:2b

# echo "id:00:01:00:01:1a:ff:2b:ff:52:54:00:3b:9a:2b,[2001:db8:aa::1010]" \
      >> /var/lib/dnsmasq/virbr10/hostsfile
# killall -SIGHUP -e dnsmasq

Add some firewall rules for both iptables and ip6tables.

*filter
... snipped ...
# Accept DNS (port 53) and DHCP (port 67) packets from VMs.
-A INPUT -i virbr10 -p udp -m udp -m multiport --dports 53,67 -j ACCEPT
-A INPUT -i virbr10 -p tcp -m tcp -m multiport --dports 53,67 -j ACCEPT

# If using DHCPv6 instead of SLAAC, also allow UDP port 547.
# -A INPUT -i virbr10 -p udp -m udp --dport 547 -j ACCEPT
... snipped ...
COMMIT

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# DHCP packets sent to VMs have no checksum (due to a longstanding bug).
-A POSTROUTING -o virbr10 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT

If you are running a system-wide instance of dnsmasq, you may need to configure it to ignore the virtual bridge.

# touch /etc/dnsmasq.d/virbr10.conf
# echo "except-interface=virbr10" >> /etc/dnsmasq.d/virbr10.conf
# echo "bind-interfaces" >> /etc/dnsmasq.d/virbr10.conf
# service dnsmasq restart

If systemd is available, Run dnsmasq with systemd. Otherwise, just run dnsmasq from the command-line:

# dnsmasq --conf-file=/var/lib/dnsmasq/virbr10/dnsmasq.conf \
      --pid-file=/var/run/dnsmasq/virbr10.pid

Configure virtual machines¶

New VM¶

# virt-install --network bridge=virbr10 ...

Optionally, pass --network more than once to create additional virtual Ethernet interfaces for the VM.

# virt-install --network network=virbr10 --network network=virbr11 ...

Existing VM¶

Open the XML configuration for the VM in a text editor.

# virsh edit name-of-vm

There should already be an <interface> section that configures a virtual Ethernet interface for the VM. Note down the MAC address.

<interface type="network">
   <source network="default"/>
   <mac address="52:54:00:4f:47:f2"/>
</interface>

To reconfigure the virtual Ethernet interface, replace the <interface> section with the following contents. Use the MAC address noted above, otherwise the MAC address of the VM will change.

<interface type="bridge">
  <source bridge="virbr10"/>
  <mac address="52:54:00:4f:47:f2"/>
</interface>

To add an additional virtual Ethernet interface, append a new <interface> section. libvirt generates a random MAC for the new interface if <mac> is omitted.

<interface type="network">
   <source network="virbr10"/>
</interface>

Reboot the VM to apply the changes. You may need to amend the VM’s network initialization scripts to account for the network interface changes.

Custom NAT-based network¶

The default NAT-based network has some limitations. Follow the steps below to overcome these limitations and take control of your server environment. There are four main components: a dummy network interface, a virtual bridge, some iptables rules, and dnsmasq.

Note

If you are uncomfortable with iptables, you might prefer to stick with the default NAT-based network.

Disable the default network¶

To prevent libvirt from altering the firewall, stop and disable the default network. Make sure there are no active virtual machines (VMs) still using this network.

# virsh net-destroy default
# virsh net-autostart --disable default

Create a dummy interface¶

Choose a MAC address for the virtual bridge. Use hexdump to generate a random MAC address in the format that libvirt expects (52:54:00:xx:xx:xx for KVM, 00:16:3e:xx:xx:xx for Xen).

# hexdump -vn3 -e '/3 "52:54:00"' -e '/1 ":%02x"' -e '"\n"' /dev/urandom
52:54:00:7e:27:af

Create a dummy network interface called virbr10-dummy and set the MAC address to the one generated above.

# ip link add virbr10-dummy address 52:54:00:7e:27:af type dummy

To create a persistent dummy interface at every boot, follow the instructions for Red Hat Enterprise Linux, Fedora, or Debian.

Create a virtual bridge¶

A Linux bridge without a real Ethernet device is considered virtual. Choose a name and a private subnet for the virtual bridge. In this example:

The virtual bridge is called virbr10.
The private subnet chosen is 192.168.100.0/24 (see CIDR notation).
- VMs can bind to addresses from 192.168.100.2 to 192.168.100.254.
- VMs see the libvirt server as 192.168.100.1.

To manually create the virtual bridge, run these commands:

# brctl addbr virbr10
# brctl stp virbr10 on
# brctl addif virbr10 virbr10-dummy
# ip address add 192.168.100.1/24 dev virbr10 broadcast 192.168.100.255

Rather than creating the bridge manually, follow the instructions for Red Hat Enterprise Linux, Fedora, or Debian (so that the bridge is created at every boot).

Implement NAT with iptables¶

IP masquerading allows many machines with private IP addresses (eg, 192.168.X.X) to communicate with the Internet through the public IP address of a router. In this case, the libvirt server acts as a router for the VMs.

A router must be able to forward network packets between interfaces, so modify some kernel parameters to allow this.

# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf
# echo "net.ipv4.conf.all.forwarding=1" >> /etc/sysctl.conf
# sysctl -p

Implement IP masquerading in the nat table.

# This format is understood by iptables-restore. See `man iptables-restore`.
*nat
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Do not masquerade to these reserved address blocks.
-A POSTROUTING -s 192.168.100.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.100.0/24 -d 255.255.255.255/32 -j RETURN
# Masquerade all packets going from VMs to the LAN/Internet.
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE
COMMIT

Allow forwarding in the filter table.

# This format is understood by iptables-restore. See `man iptables-restore`.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]

... snipped ...
# Allow established traffic to the private subnet.
-A FORWARD -d 192.168.100.0/24 -o virbr10 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow outbound traffic from the private subnet.
-A FORWARD -s 192.168.100.0/24 -i virbr10 -j ACCEPT
# Allow traffic between virtual machines.
-A FORWARD -i virbr10 -o virbr10 -j ACCEPT
# Reject everything else.
-A FORWARD -i virbr10 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o virbr10 -j REJECT --reject-with icmp-port-unreachable
... snipped ...
COMMIT

See Example of iptables NAT for a full iptables rule set.

Run dnsmasq¶

Note

This step is optional, as VMs can bind to addresses without DHCP and can use their own DNS resolver. However, it is recommended unless you know what you are doing.

On the libvirt server, it is common to run a DHCP server (to decide which IP address to lease to each VM) and a DNS server (to respond to queries from VMs). dnsmasq is ideal for both of these tasks.

Create files and directories needed for dnsmasq.

# mkdir -p /var/lib/dnsmasq/virbr10
# touch /var/lib/dnsmasq/virbr10/hostsfile
# touch /var/lib/dnsmasq/virbr10/leases

Create /var/lib/dnsmasq/virbr10/dnsmasq.conf with these contents:

# Only bind to the virtual bridge. This avoids conflicts with other running
# dnsmasq instances.
except-interface=lo
interface=virbr10
bind-dynamic

# If using dnsmasq 2.62 or older, remove "bind-dynamic" and "interface" lines
# and uncomment these lines instead:
#bind-interfaces
#listen-address=192.168.100.1

# IPv4 addresses to offer to VMs. This should match the chosen subnet.
dhcp-range=192.168.100.2,192.168.100.254

# Set this to at least the total number of addresses in DHCP-enabled subnets.
dhcp-lease-max=1000

# File to write DHCP lease information to.
dhcp-leasefile=/var/lib/dnsmasq/virbr10/leases
# File to read DHCP host information from.
dhcp-hostsfile=/var/lib/dnsmasq/virbr10/hostsfile
# Avoid problems with old or broken clients.
dhcp-no-override
# https://www.redhat.com/archives/libvir-list/2010-March/msg00038.html
strict-order

Optionally, use hostsfile to always assign a specific IP address to a VM with a specific MAC address. (dnsmasq only reads changes after receiving a SIGHUP.)

# echo "52:54:00:be:0a:f3,192.168.100.77" \
      >> /var/lib/dnsmasq/virbr10/hostsfile

Add some iptables rules. (See Example of iptables NAT for a full iptables rule set.)

*filter
... snipped ...
# Accept DNS (port 53) and DHCP (port 67) packets from VMs.
-A INPUT -i virbr10 -p udp -m udp -m multiport --dports 53,67 -j ACCEPT
-A INPUT -i virbr10 -p tcp -m tcp -m multiport --dports 53,67 -j ACCEPT
... snipped ...
COMMIT

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# DHCP packets sent to VMs have no checksum (due to a longstanding bug).
-A POSTROUTING -o virbr10 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT

If you are running a system-wide instance of dnsmasq, you may need to configure it to ignore the virtual bridge.

# touch /etc/dnsmasq.d/virbr10.conf
# echo "except-interface=virbr10" >> /etc/dnsmasq.d/virbr10.conf
# echo "bind-interfaces" >> /etc/dnsmasq.d/virbr10.conf
# service dnsmasq restart

If systemd is available, Run dnsmasq with systemd. Otherwise, just run dnsmasq from the command-line:

# dnsmasq --conf-file=/var/lib/dnsmasq/virbr10/dnsmasq.conf \
      --pid-file=/var/run/dnsmasq/virbr10.pid

Forward incoming connections¶

Note

This step is optional. It is only necessary if one or more VMs are running services (eg, web applications) that need to be available over the network.

If one of the VMs has a web application listening on ports 80/443, connections to those ports on the server (eg, 203.0.113.3) can be forwarded to the VM (eg, 192.168.100.77). Add these iptables rules:

*nat
... snipped ...
# Modify the destination address of packets received on ports 80 and 443.
-A PREROUTING -d 203.0.113.3/32 -p tcp -m tcp --syn -m multiport --dports 80,443 -j DNAT --to-destination 192.168.100.77

# Optionally, make the VM accessible via `ssh -p 2222 [email protected]`.
-A PREROUTING -d 203.0.113.3/32 -p tcp -m tcp --syn --dport 2222 -j DNAT --to-destination 192.168.100.77:22
COMMIT

*filter
... snipped ...
# Allow packets that have been forwarded to particular ports on the VM.
-A FORWARD -d 192.168.100.77/32 -o virbr10 -p tcp -m tcp --syn -m conntrack --ctstate NEW -m multiport --dports 22,80,443 -j ACCEPT
... snipped ...
COMMIT

See Example of iptables NAT with connection forwarding for a full iptables rule set.

Configure virtual machines¶

New VM¶

# virt-install --network bridge=virbr10 ...

Optionally, pass --network more than once to create additional virtual Ethernet interfaces for the VM.

# virt-install --network network=virbr10 --network network=virbr11 ...

Existing VM¶

Open the XML configuration for the VM in a text editor.

# virsh edit name-of-vm

There should already be an <interface> section that configures a virtual Ethernet interface for the VM. Note down the MAC address.

<interface type="network">
   <source network="default"/>
   <mac address="52:54:00:4f:47:f2"/>
</interface>

To reconfigure the virtual Ethernet interface, replace the <interface> section with the following contents. Use the MAC address noted above, otherwise the MAC address of the VM will change.

<interface type="bridge">
  <source bridge="virbr10"/>
  <mac address="52:54:00:4f:47:f2"/>
</interface>

To add an additional virtual Ethernet interface, append a new <interface> section. libvirt generates a random MAC for the new interface if <mac> is omitted.

<interface type="network">
   <source network="virbr10"/>
</interface>

Reboot the VM to apply the changes. You may need to amend the VM’s network initialization scripts to account for the network interface changes.

Multiple networks¶

Sometimes it makes sense to run multiple networks of different types for your virtual machines (VMs). libvirt allows each VM to have several network interface controllers (NICs), each connected to a different network.

Example 1¶

VMs on a dedicated server need to serve clients over both IPv4 and IPv6. The hosting provider has allocated a /56 block of IPv6 addresses but only one IPv4 address. Due to hosting provider limitations, bridging is unavailable.

The first NIC connects to a Routed network and allows VMs to have their own (statically routed) IPv6 address.
The second NIC connects to a NAT-based network and provides IPv4 connectivity without having to purchase additional IPv4 addresses.

Example 2¶

VMs on a dedicated server need to serve clients over both IPv4 and IPv6. The hosting provider has allocated a /64 block of IPv6 addresses but only one IPv4 address. The dedicated server is on a private VLAN that allows VMs to bind directly to IP addresses.

The first NIC connects to a Bridged network and allows VMs to bind directly to IPv6 addresses on the VLAN.
The second NIC connects to a NAT-based network and provides IPv4 connectivity without having to purchase additional IPv4 addresses.

Example 3¶

The libvirt server in a home lab is connected to one network via Ethernet and one network via wireless. VMs are running services that must be available to clients on both of these networks.

The first NIC connects to a Bridged network and allows clients on the Ethernet network to access VMs.
The seconds NIC connects to a Routed network and allows clients on the wireless network to access VMs.

Dummy interface on RHEL¶

With NetworkManager

Create /etc/NetworkManager/dispatcher.d/99-virbr10 and make it executable. This script configures the dummy interface when the virtual bridge is brought up. Use the MAC address that you chose earlier for the virtual bridge.

#!/bin/sh
# See the "DISPATCHER SCRIPTS" section of `man NetworkManager`.
# Remember to make this file executable!
[ "$1" != "virbr10" ] && exit 0
case "$2" in
    "up")
        # Create the dummy interface.
        /sbin/ip link add virbr10-dummy address 52:54:00:7e:27:af type dummy
        # Attach the dummy interface to the bridge.
        /usr/sbin/brctl addif virbr10 virbr10-dummy
        ;;
esac

Without NetworkManager

Create /etc/sysconfig/virbr10-dummy. Use the MAC address that you chose earlier for the virtual bridge.

# echo "MACADDR=52:54:00:7e:27:af" > /etc/sysconfig/virbr10-dummy

Create /etc/systemd/system/dummy@.service with these contents:

# '%i' becomes 'virbr10' when running `systemctl start [email protected]`
# Remember to run `systemctl daemon-reload` after creating or editing this file.

[Unit]
Description=Dummy network interface for %i
After=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=/etc/sysconfig/%i-dummy
ExecStartPre=-/sbin/ip link add %i-dummy address ${MACADDR} type dummy
ExecStart=/usr/sbin/brctl addif %i %i-dummy

[Install]
WantedBy=multi-user.target

Enable and start the service.

# systemctl daemon-reload
# systemctl enable [email protected]
# systemctl start [email protected]

Without NetworkManager and without systemd

Warning

This requires loading the dummy kernel module, which creates an interface called dummy0. When following instructions later in this guide, remember that the interface is called dummy0, not virbr10-dummy, and amend the commands accordingly.

Create /etc/sysconfig/network-scripts/ifcfg-dummy0. Use the MAC address that you chose earlier for the virtual bridge.

DEVICE=dummy0
MACADDR=52:54:00:7e:27:af
NM_CONTROLLED=no
ONBOOT=yes
TYPE=Ethernet
BRIDGE=virbr10
IPV6INIT=no

Load the dummy module.

# modprobe dummy numdummies=1
# echo "dummy" > /etc/modules-load.d/dummy.conf
# echo "options dummy numdummies=1" > /etc/modprobe.d/dummy.conf

If using Red Hat Enterprise Linux 6, create an executable file called /etc/sysconfig/modules/dummy.modules with the following contents:

#!/bin/sh
/sbin/modprobe dummy numdummies=1

Dummy interface on Fedora¶

With NetworkManager

#!/bin/sh
# See the "DISPATCHER SCRIPTS" section of `man NetworkManager`.
# Remember to make this file executable!
[ "$1" != "virbr10" ] && exit 0
case "$2" in
    "up")
        # Create the dummy interface.
        /sbin/ip link add virbr10-dummy address 52:54:00:7e:27:af type dummy
        # Attach the dummy interface to the bridge.
        /usr/sbin/brctl addif virbr10 virbr10-dummy
        ;;
esac

Without NetworkManager

Create /etc/sysconfig/virbr10-dummy. Use the MAC address that you chose earlier for the virtual bridge.

# echo "MACADDR=52:54:00:7e:27:af" > /etc/sysconfig/virbr10-dummy

Create /etc/systemd/system/dummy@.service with these contents:

# '%i' becomes 'virbr10' when running `systemctl start [email protected]`
# Remember to run `systemctl daemon-reload` after creating or editing this file.

[Unit]
Description=Dummy network interface for %i
After=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=/etc/sysconfig/%i-dummy
ExecStartPre=-/sbin/ip link add %i-dummy address ${MACADDR} type dummy
ExecStart=/usr/sbin/brctl addif %i %i-dummy

[Install]
WantedBy=multi-user.target

Enable and start the service.

# systemctl daemon-reload
# systemctl enable [email protected]
# systemctl start [email protected]

Dummy interface on Debian¶

Append the following text to /etc/network/interfaces. Use the MAC address that you chose earlier for the virtual bridge.

auto virbr10-dummy
iface virbr10-dummy inet manual
    pre-up /sbin/ip link add virbr10-dummy type dummy
    up /sbin/ip link set virbr10-dummy address 52:54:00:7e:27:af

Routed virtual bridge on RHEL¶

Create /etc/sysconfig/network-scripts/ifcfg-virbr10 with these contents:

DEVICE=virbr10
NAME=virbr10
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
DELAY=2
STP=on

IPADDR=203.0.113.88
NETMASK=255.255.255.248

IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6ADDR=2001:db8:aa::1
IPV6FORWARDING=yes

If using NetworkManager:

# nmcli connection load /etc/sysconfig/network-scripts/ifcfg-virbr10
# nmcli connection up virbr10

If using NetworkManager on Red Hat Enterprise Linux 6:

# echo "NM_BOND_BRIDGE_VLAN_ENABLED=yes" >> /etc/sysconfig/network
# service NetworkManager restart

If not using NetworkManager:

# ifup virbr10
# ifup virbr10-dummy

Routed virtual bridge on Fedora¶

Create /etc/sysconfig/network-scripts/ifcfg-virbr10 with these contents:

DEVICE=virbr10
NAME=virbr10
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
DELAY=2
STP=on

IPADDR=203.0.113.88
NETMASK=255.255.255.248

IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6ADDR=2001:db8:aa::1
IPV6FORWARDING=yes

If using NetworkManager:

# nmcli connection load /etc/sysconfig/network-scripts/ifcfg-virbr10
# nmcli connection up virbr10

If not using NetworkManager:

# ifup virbr10
# ifup virbr10-dummy

Routed virtual bridge on Debian¶

Append the following text to /etc/network/interfaces:

auto virbr10
iface virbr10 inet static
    # Make sure bridge-utils is installed!
    bridge_ports virbr10-dummy
    bridge_stp on
    bridge_fd 2
    address 203.0.113.88
    netmask 255.255.255.248

iface virbr10 inet6 static
    address 2001:db8:aa::1
    netmask 64

Bring the virtual bridge up.

# ifup virbr10-dummy
# ifup virbr10

NAT virtual bridge on RHEL¶

Create /etc/sysconfig/network-scripts/ifcfg-virbr10 with these contents:

DEVICE=virbr10
NAME=virbr10
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
DELAY=2
STP=on
IPADDR=192.168.100.1
NETMASK=255.255.255.0
IPV6INIT=no

If using NetworkManager:

# nmcli connection load /etc/sysconfig/network-scripts/ifcfg-virbr10
# nmcli connection up virbr10

If using NetworkManager on Red Hat Enterprise Linux 6:

# echo "NM_BOND_BRIDGE_VLAN_ENABLED=yes" >> /etc/sysconfig/network
# service NetworkManager restart
# nmcli con up id virbr10

If not using NetworkManager:

# ifup virbr10
# ifup virbr10-dummy

NAT virtual bridge on Fedora¶

Create /etc/sysconfig/network-scripts/ifcfg-virbr10 with these contents:

DEVICE=virbr10
NAME=virbr10
# Change to 'no' to disable NetworkManager for this interface.
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Bridge
DELAY=2
STP=on
IPADDR=192.168.100.1
NETMASK=255.255.255.0
IPV6INIT=no

If using NetworkManager:

# nmcli connection load /etc/sysconfig/network-scripts/ifcfg-virbr10
# nmcli connection up virbr10

If not using NetworkManager:

# ifup virbr10
# ifup virbr10-dummy

NAT virtual bridge on Debian¶

Append the following text to /etc/network/interfaces:

auto virbr10
iface virbr10 inet static
    # Make sure bridge-utils is installed!
    bridge_ports virbr10-dummy
    bridge_stp on
    bridge_fd 2
    address 192.168.100.1
    netmask 255.255.255.0

Bring the virtual bridge up.

# ifup virbr10-dummy
# ifup virbr10

Run dnsmasq with systemd¶

Create /etc/systemd/system/dnsmasq@.service with these contents:

# '%i' becomes 'virbr10' when running `systemctl start [email protected]`
# Remember to run `systemctl daemon-reload` after creating or editing this file.

[Unit]
Description=DHCP and DNS caching server for %i.
After=network.target

[Service]
ExecStart=/usr/sbin/dnsmasq -k --conf-file=/var/lib/dnsmasq/%i/dnsmasq.conf
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

It is best to only start dnsmasq when the bridge is up. If using NetworkManager, create /etc/NetworkManager/dispatcher.d/99-virbr10 and make it executable.

#!/bin/sh
# See the "DISPATCHER SCRIPTS" section of `man NetworkManager`.
# Remember to make this file executable!
[ "$1" != "virbr10" ] && exit 0
case "$2" in
    "up")
        /bin/systemctl start [email protected] || :
        ;;
    "down")
        /bin/systemctl stop [email protected] || :
        ;;
esac

If using Debian, append two command options to /etc/network/interfaces.

auto virbr10
iface virbr10 inet static
    ... snipped ...
    up /bin/systemctl start [email protected] || :
    down /bin/systemctl stop [email protected] || :

Alternatively, just start dnsmasq at every boot.

# systemctl enable [email protected]
# systemctl start [email protected]

DHCP host entries¶

It is often useful to always assign a specific IPv4 or IPv6 address to a particular VM. Open the network XML configuration in a text editor. For example, to edit the default network:

# virsh net-edit default

For IPv4, use the VM’s MAC address to add a DHCPv4 <host> element. Add as many <host> elements as required.

<network>
  <name>default</name>
  <bridge name="virbr0"/>
  <forward mode="nat"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
      <host mac="52:54:00:6f:78:f3" ip="192.168.122.222"/>
      <host mac="52:54:00:91:ed:23" ip="192.168.122.233"/>
    </dhcp>
  </ip>
</network>

For IPv6, MAC addresses are deprecated so use the VM’s DHCP Unique Identifier (DUID) instead. To find the DUID for a running VM, use virsh.

# virsh net-dhcp-leases routed

MAC address        Client ID or DUID
------------------------------------
52:54:00:8a:11:4c  00:01:00:01:1a:ff:2b:ff:52:54:00:8a:11:4c
52:54:00:9b:ae:41  00:01:00:01:1a:ff:2b:ff:52:54:00:9b:ae:41

Use the VM’s DUID to add a DHCPv6 <host> element. Add as many <host> elements as required.

<network>
  <name>default</name>
  <bridge name="virbr0"/>
  <forward mode="nat"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="2001:db8::1" prefix="64">
    <dhcp>
      <range start="2001:db8::1000" end="2001:db8::1fff"/>
      <host id="00:01:00:01:1a:ff:2b:ff:52:54:00:8a:11:4c" ip="2001:db8::1222"/>
      <host id="00:01:00:01:1a:ff:2b:ff:52:54:00:9b:ae:41" ip="2001:db8::1333"/>
    </dhcp>
  </ip>
</network>

Restart the virtual network.

# virsh net-destroy default
# virsh net-start default

Without restarting the virtual network

It is a bit tiresome to restart the virtual network after every change. A much better approach is to use virsh net-update, which makes live changes to an active virtual network. (--parent-index specifies which element to select if there is more than one <ip> element.)

# virsh net-update default add-last ip-dhcp-host \
      '<host mac="52:54:00:6f:78:f3" ip="192.168.122.222"/>' \
      --live --config --parent-index 0

# virsh net-update default add-last ip-dhcp-host \
      '<host id="00:01:00:01:1a:ff:2b:ff:52:54:00:8a:11:4c" ip="2001:db8::1222"/>' \
      --live --config --parent-index 1

To modify a DHCP host entry on an active network:

# virsh net-update default modify ip-dhcp-host \
      '<host mac="52:54:00:6f:78:f3" ip="192.168.122.222"/>' \
      --live --config --parent-index 0

To delete a DHCP host entry on an active network:

# virsh net-update default delete ip-dhcp-host \
      '<host mac="52:54:00:6f:78:f3" ip="192.168.122.222"/>' \
      --live --config --parent-index 0

Run radvd¶

If using a version of dnsmasq older than 2.64, stateless address autoconfiguration (SLAAC) may not work as expected and VMs will not receive an IPv6 address. In this situation, use the Router Advertisement Daemon (radvd) instead, which can be run alongside dnsmasq.

Open /etc/radvd.conf in a text editor and add the following content. Add as many interface sections as required, then start the radvd service.

interface virbr10
{
  AdvSendAdvert on;
  AdvManagedFlag off;
  AdvOtherConfigFlag off;

  prefix 2001:db8:aa::/64
  {
    AdvOnLink on;
    AdvAutonomous on;
    AdvRouterAddr off;
  };
};

Example of iptables NAT¶

If using Red Hat Enterprise Linux (or Fedora), install iptables and save the rules below as /etc/sysconfig/iptables.

# yum install iptables-services
# service iptables enable

If using Debian, install iptables and save the rules below as /etc/iptables/rules.v4.

# apt-get install iptables-persistent
# update-rc.d netfilter-persistent enable

In this example:

The virtual bridge is called virbr10.
The private subnet chosen is 192.168.100.0/24.
- VMs can bind to addresses from 192.168.100.2 to 192.168.100.254.
- VMs see the libvirt server as 192.168.100.1.

Note

This format is understood by iptables-restore. See man iptables-restore.

# This format is understood by iptables-restore. See `man iptables-restore`.

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# DHCP packets sent to VMs have no checksum (due to a longstanding bug).
-A POSTROUTING -o virbr10 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Do not masquerade to these reserved address blocks.
-A POSTROUTING -s 192.168.100.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.100.0/24 -d 255.255.255.255/32 -j RETURN
# Masquerade all packets going from VMs to the LAN/Internet.
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE
COMMIT

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# Allow basic INPUT traffic.
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
# Accept SSH connections.
-A INPUT -p tcp -m tcp --syn -m conntrack --ctstate NEW --dport 22 -j ACCEPT
# Accept DNS (port 53) and DHCP (port 67) packets from VMs.
-A INPUT -i virbr10 -p udp -m udp -m multiport --dports 53,67 -j ACCEPT
-A INPUT -i virbr10 -p tcp -m tcp -m multiport --dports 53,67 -j ACCEPT
# Reject everything else.
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p tcp -m tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-port-unreachable

# Allow established traffic to the private subnet.
-A FORWARD -d 192.168.100.0/24 -o virbr10 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow outbound traffic from the private subnet.
-A FORWARD -s 192.168.100.0/24 -i virbr10 -j ACCEPT
# Allow traffic between virtual machines.
-A FORWARD -i virbr10 -o virbr10 -j ACCEPT
# Reject everything else.
-A FORWARD -i virbr10 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o virbr10 -j REJECT --reject-with icmp-port-unreachable
COMMIT

Example of iptables NAT with connection forwarding¶

If using Red Hat Enterprise Linux (or Fedora), install iptables and save the rules below as /etc/sysconfig/iptables.

# yum install iptables-services
# service iptables enable

If using Debian, install iptables and save the rules below as /etc/iptables/rules.v4.

# apt-get install iptables-persistent
# update-rc.d netfilter-persistent enable

In this example:

The virtual bridge is called virbr10.
The private subnet chosen is 192.168.100.0/24.
- VMs can bind to addresses from 192.168.100.2 to 192.168.100.254.
- VMs see the libvirt server as 192.168.100.1.
The libvirt server has public IP address 203.0.113.3.
The target VM has private IP address 192.168.100.77.
- Connections to port 80/443 on the server are forwarded to the target VM.
- Connections to port 7722 on the server are forwarded to port 22 on the target VM.

# This format is understood by iptables-restore. See `man iptables-restore`.

*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# DHCP packets sent to VMs have no checksum (due to a longstanding bug).
-A POSTROUTING -o virbr10 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT

*nat
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# Modify the destination address of packets received on ports 80 and 443.
-A PREROUTING -d 203.0.113.3/32 -p tcp -m tcp --syn -m multiport --dports 80,443 -j DNAT --to-destination 192.168.100.77
# Optionally, make the VM accessible via `ssh -p 7722 user@203.0.113.3`.
-A PREROUTING -d 203.0.113.3/32 -p tcp -m tcp --syn --dport 7722 -j DNAT --to-destination 192.168.100.77:22

# Do not masquerade to these reserved address blocks.
-A POSTROUTING -s 192.168.100.0/24 -d 224.0.0.0/24 -j RETURN
-A POSTROUTING -s 192.168.100.0/24 -d 255.255.255.255/32 -j RETURN
# Masquerade all packets going from VMs to the LAN/Internet.
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE
COMMIT

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
# Allow basic INPUT traffic.
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
# Accept SSH connections.
-A INPUT -p tcp -m tcp --syn -m conntrack --ctstate NEW --dport 22 -j ACCEPT
# Accept DNS (port 53) and DHCP (port 67) packets from VMs.
-A INPUT -i virbr10 -p udp -m udp -m multiport --dports 53,67 -j ACCEPT
-A INPUT -i virbr10 -p tcp -m tcp -m multiport --dports 53,67 -j ACCEPT
# Reject everything else.
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p tcp -m tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-port-unreachable

# Allow established traffic to the private subnet.
-A FORWARD -d 192.168.100.0/24 -o virbr10 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Allow outbound traffic from the private subnet.
-A FORWARD -s 192.168.100.0/24 -i virbr10 -j ACCEPT
# Allow traffic between virtual machines.
-A FORWARD -i virbr10 -o virbr10 -j ACCEPT
# Allow packets that have been forwarded to particular ports on the VM.
-A FORWARD -d 192.168.100.77/32 -o virbr10 -p tcp -m tcp --syn -m conntrack --ctstate NEW -m multiport --dports 22,80,443 -j ACCEPT
# Reject everything else.
-A FORWARD -i virbr10 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o virbr10 -j REJECT --reject-with icmp-port-unreachable
COMMIT

Last update: Jun 30, 2025