[libvirt] [PATCH] nwfilter: Support for learning a VM's IP address

Subject: Support for learning a VM's IP address This patch implements support for learning a VM's IP address. It uses the pcap library to listen on the VM's backend network interface (tap) or the physical ethernet device (macvtap) and tries to capture packets with source or destination MAC address of the VM and learn from DHCP Offers, ARP traffic, or first-sent IPv4 packet what the IP address of the VM's interface is. This then allows to instantiate the network traffic filtering rules without the user having to provide the IP parameter somewhere in the filter description or in the interface description as a parameter. This only supports to detect the parameter IP, which is for the assumed single IPv4 address of a VM. There is not support for interfaces that may have multiple IP addresses (IP aliasing) or IPv6 that may then require more than one valid IP address to be detected. A VM can have multiple independent interfaces that each uses a different IP address and in that case it will be attempted to detect each one of the address independently. So, when for example an interface description in the domain XML has looked like this up to now: <interface type='bridge'> <source bridge='mybridge'/> <model type='virtio'/> <filterref filter='clean-traffic'> <parameter name='IP' value='10.2.3.4'/> </filterref> </interface> you may omit the IP parameter: <interface type='bridge'> <source bridge='mybridge'/> <model type='virtio'/> <filterref filter='clean-traffic'/> </interface> Internally I am walking the 'tree' of a VM's referenced network filters and determine with the given variables which variables are missing. Now, the above IP parameter may be missing and this causes a libvirt-internal thread to be started that uses the pcap library's API to listen to the backend interface (in case of macvtap to the physical interface) in an attempt to determine the missing IP parameter. If the backend interface disappears the thread terminates assuming the VM was brought down. In case of a macvtap device a timeout is being used to wait for packets from the given VM (filtering by VM's interface MAC address). If the VM's macvtap device disappeared the thread also terminates. In all other cases it tries to determine the IP address of the VM and will then apply the rules late on the given interface, which would have happened immediately if the IP parameter had been explicitly given. In case an error happens while the firewall rules are applied, the VM's backend interface is 'down'ed preventing it to communicate. Reasons for failure for applying the network firewall rules may that an ebtables/iptables command failes or OOM errors. Essentially the same failure reasons may occur as when the firewall rules are applied immediately on VM start, except that due to the late application of the filtering rules the VM now is already running and cannot be hindered anymore from starting. Bringing down the whole VM would probably be considered too drastic. While a VM's IP address is attempted to be determined only limited updates to network filters are allowed. In particular it is prevented that filters are modified in such a way that they would introduce new variables. A caveat: The algorithm does not know which one is the appropriate IP address of a VM. If the VM spoofs an IP address in its first ARP traffic or IPv4 packets its filtering rules will be instantiated for this IP address, thus 'locking' it to the found IP address. So, it's still 'safer' to explicitly provide the IP address of a VM's interface in the filter description if it is known beforehand. Signed-off-by: Stefan Berger <stefanb@us.ibm.com> --- configure.ac | 39 ++ src/Makefile.am | 13 src/conf/nwfilter_conf.c | 7 src/conf/nwfilter_conf.h | 3 src/libvirt_private.syms | 12 src/nwfilter/nwfilter_driver.c | 11 src/nwfilter/nwfilter_ebiptables_driver.c | 100 +++++ src/nwfilter/nwfilter_ebiptables_driver.h | 7 src/nwfilter/nwfilter_gentech_driver.c | 408 +++++++++++++++++++-- src/nwfilter/nwfilter_gentech_driver.h | 14 src/nwfilter/nwfilter_learnipaddr.c | 579 ++++++++++++++++++++++++++++++ src/nwfilter/nwfilter_learnipaddr.h | 61 +++ 12 files changed, 1216 insertions(+), 38 deletions(-) Index: libvirt-acl/src/nwfilter/nwfilter_learnipaddr.h =================================================================== --- /dev/null +++ libvirt-acl/src/nwfilter/nwfilter_learnipaddr.h @@ -0,0 +1,61 @@ +/* + * nwfilter_learnipaddr.h: support for learning IP address used by a VM + * on an interface + * + * Copyright (C) 2010 IBM Corp. + * Copyright (C) 2010 Stefan Berger + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Stefan Berger <stefanb@us.ibm.com> + */ + +#ifndef __NWFILTER_LEARNIPADDR_H +#define __NWFILTER_LEARNIPADDR_H + +typedef struct _virNWFilterIPAddrLearnReq virNWFilterIPAddrLearnReq; +typedef virNWFilterIPAddrLearnReq *virNWFilterIPAddrLearnReqPtr; +struct _virNWFilterIPAddrLearnReq { + char ifname[IF_NAMESIZE]; + char linkdev[IF_NAMESIZE]; + enum virDomainNetType nettype; + unsigned char macaddr[VIR_MAC_BUFLEN]; + char *filtername; + virNWFilterHashTablePtr filterparams; + virNWFilterDriverStatePtr driver; + + int status; + pthread_t thread; +}; + +int virNWFilterLearnIPAddress(virConnectPtr conn, + const char *ifname, + const char *linkdev, + enum virDomainNetType nettype, + const unsigned char *macaddr, + const char *filtername, + virNWFilterHashTablePtr filterparams, + virNWFilterDriverStatePtr driver); + +virNWFilterIPAddrLearnReqPtr virNWFilterLookupLearnReq(const char *ifname); + + +void virNWFilterDelIpAddrForIfname(const char *ifname); +const char *virNWFilterGetIpAddrForIfname(const char *ifname); + +int virNWFilterLearnInit(void); +void virNWFilterLearnShutdown(void); + +#endif /* __NWFILTER_LEARNIPADDR_H */ Index: libvirt-acl/src/nwfilter/nwfilter_learnipaddr.c =================================================================== --- /dev/null +++ libvirt-acl/src/nwfilter/nwfilter_learnipaddr.c @@ -0,0 +1,579 @@ +/* + * nwfilter_learnipaddr.c: support for learning IP address used by a VM + * on an interface + * + * Copyright (C) 2010 IBM Corp. + * Copyright (C) 2010 Stefan Berger + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + * + * Author: Stefan Berger <stefanb@us.ibm.com> + */ + +#include <config.h> + +#ifdef HAVE_LIBPCAP +#include <pcap.h> +#endif + +#include <fcntl.h> +#include <sys/ioctl.h> + +#include <arpa/inet.h> +#include <net/ethernet.h> +#include <netinet/ip.h> +#include <netinet/udp.h> +#include <net/if_arp.h> +#include <linux/if.h> + +#include "internal.h" + +#include "buf.h" +#include "memory.h" +#include "logging.h" +#include "datatypes.h" +#include "virterror_internal.h" +#include "threads.h" +#include "conf/nwfilter_params.h" +#include "conf/domain_conf.h" +#include "nwfilter_gentech_driver.h" +#include "nwfilter_ebiptables_driver.h" +#include "nwfilter_learnipaddr.h" + +#define VIR_FROM_THIS VIR_FROM_NWFILTER + + +/* structure of an ARP request/reply message */ +struct f_arphdr { + struct arphdr arphdr; + uint8_t ar_sha[ETH_ALEN]; + uint32_t ar_sip; + uint8_t ar_tha[ETH_ALEN]; + uint32_t ar_tip; +} __attribute__((packed)); + + +/* structure representing DHCP message */ +struct dhcp { + uint8_t op; + uint8_t htype; + uint8_t hlen; + uint8_t hops; + uint32_t xid; + uint16_t secs; + uint16_t flags; + uint32_t ciaddr; + uint32_t yiaddr; + uint32_t siaddr; + uint32_t giaddr; + uint32_t chaddr; + /* omitted */ +} __attribute__((packed)); + + +struct ether_vlan_header +{ + uint8_t dhost[ETH_ALEN]; + uint8_t shost[ETH_ALEN]; + uint16_t vlan_type; + uint16_t vlan_flags; + uint16_t ether_type; +} __attribute__((packed)); + + +static virMutex pendingLearnReqLock; +static virHashTablePtr pendingLearnReq; + +static virMutex ipAddressMapLock; +static virNWFilterHashTablePtr ipAddressMap; + + +static void +virNWFilterIPAddrLearnReqFree(virNWFilterIPAddrLearnReqPtr req) { + if (!req) + return; + + VIR_FREE(req->filtername); + virNWFilterHashTableFree(req->filterparams); + + VIR_FREE(req); +} + + +#if HAVE_LIBPCAP + +static int +virNWFilterRegisterLearnReq(virNWFilterIPAddrLearnReqPtr req) { + int res = -1; + virMutexLock(&pendingLearnReqLock); + + if (!virHashLookup(pendingLearnReq, req->ifname)) + res = virHashAddEntry(pendingLearnReq, req->ifname, req); + + virMutexUnlock(&pendingLearnReqLock); + + return res; +} + +#endif + + +virNWFilterIPAddrLearnReqPtr +virNWFilterLookupLearnReq(const char *ifname) { + void *res; + + virMutexLock(&pendingLearnReqLock); + + res = virHashLookup(pendingLearnReq, ifname); + + virMutexUnlock(&pendingLearnReqLock); + + return res; +} + + +static void +freeLearnReqEntry(void *payload, const char *name ATTRIBUTE_UNUSED) { + virNWFilterIPAddrLearnReqFree(payload); +} + + +#ifdef HAVE_LIBPCAP + +static virNWFilterIPAddrLearnReqPtr +virNWFilterDeregisterLearnReq(const char *ifname) { + virNWFilterIPAddrLearnReqPtr res; + + virMutexLock(&pendingLearnReqLock); + + res = virHashLookup(pendingLearnReq, ifname); + + if (res) + virHashRemoveEntry(pendingLearnReq, ifname, NULL); + + virMutexUnlock(&pendingLearnReqLock); + + return res; +} + + + +static int +virNWFilterAddIpAddrForIfname(const char *ifname, char *addr) { + int ret; + + virMutexLock(&ipAddressMapLock); + + ret = virNWFilterHashTablePut(ipAddressMap, ifname, addr, 1); + + virMutexUnlock(&ipAddressMapLock); + + return ret; +} +#endif + + +void +virNWFilterDelIpAddrForIfname(const char *ifname) { + + virMutexLock(&ipAddressMapLock); + + if (virHashLookup(ipAddressMap->hashTable, ifname)) + virNWFilterHashTableRemoveEntry(ipAddressMap, ifname); + + virMutexUnlock(&ipAddressMapLock); +} + + +const char * +virNWFilterGetIpAddrForIfname(const char *ifname) { + const char *res; + + virMutexLock(&ipAddressMapLock); + + res = virHashLookup(ipAddressMap->hashTable, ifname); + + virMutexUnlock(&ipAddressMapLock); + + return res; +} + + +#ifdef HAVE_LIBPCAP + +/** + * learnIPAddressThread + * arg: pointer to virNWFilterIPAddrLearnReq structure + * + * Learn the IP address being used on an interface. Use ARP Request and + * Reply messages, DHCP offers and the first IP packet being sent from + * the VM to detect the IP address it is using. Detects only one IP address + * per interface (IP aliasing not supported) + */ +static void * +learnIPAddressThread(void *arg) +{ + char errbuf[PCAP_ERRBUF_SIZE] = {0}; + pcap_t *handle; + struct bpf_program fp; + struct pcap_pkthdr header; + const u_char *packet; + struct ether_header *ether_hdr; + struct ether_vlan_header *vlan_hdr; + virNWFilterIPAddrLearnReqPtr req = arg; + uint32_t vmaddr = 0; + unsigned int ethHdrSize; + char *listen_if = (strlen(req->linkdev) != 0) ? req->linkdev + : req->ifname; + int to_ms = (strlen(req->linkdev) != 0) ? 1000 + : 0; + char macaddr[VIR_MAC_STRING_BUFLEN]; + virBuffer buf = VIR_BUFFER_INITIALIZER; + char *filter= NULL; + uint16_t etherType; + + req->status = 0; + + handle = pcap_open_live(listen_if, BUFSIZ, 0, to_ms, errbuf); + + if (handle == NULL) { + VIR_DEBUG("Couldn't open device %s: %s\n", listen_if, errbuf); + req->status = ENODEV; + goto done; + } + + virFormatMacAddr(req->macaddr, macaddr); + virBufferVSprintf(&buf, "ether host %s", macaddr); + if (virBufferError(&buf)) { + req->status = ENOMEM; + goto done; + } + + filter = virBufferContentAndReset(&buf); + + if (pcap_compile(handle, &fp, filter, 1, 0) != 0 || + pcap_setfilter(handle, &fp) != 0) { + VIR_DEBUG("Couldn't compile or set filter '%s'.\n", filter); + req->status = EINVAL; + goto done; + } + + while (req->status == 0 && vmaddr == 0) { + packet = pcap_next(handle, &header); + + if (!packet) { + if (to_ms == 0) { + /* assuming IF disappeared */ + req->status = ENODEV; + break; + } + /* listening on linkdev, check whether VM's dev is still there */ + if (checkIf(req->ifname, req->macaddr)) { + req->status = ENODEV; + break; + } + continue; + } + + if (header.len >= sizeof(struct ether_header)) { + ether_hdr = (struct ether_header*)packet; + + switch (ntohs(ether_hdr->ether_type)) { + + case ETHERTYPE_IP: + ethHdrSize = sizeof(struct ether_header); + etherType = ntohs(ether_hdr->ether_type); + break; + + case ETHERTYPE_VLAN: + ethHdrSize = sizeof(struct ether_vlan_header); + vlan_hdr = (struct ether_vlan_header *)packet; + if (ntohs(vlan_hdr->ether_type) != ETHERTYPE_IP || + header.len < ethHdrSize) + continue; + etherType = ntohs(vlan_hdr->ether_type); + break; + + default: + continue; + } + + if (memcmp(ether_hdr->ether_shost, + req->macaddr, + VIR_MAC_BUFLEN) == 0) { + // packets from the VM + + if ((header.len >= ethHdrSize + + sizeof(struct iphdr))) { + struct iphdr *iphdr = (struct iphdr*)(packet + + ethHdrSize); + vmaddr = iphdr->saddr; + // skip eth. bcast and mcast addresses, + // and zero address in DHCP Requests + if ((ntohl(vmaddr) & 0xc0000000) || vmaddr == 0) { + vmaddr = 0; + continue; + } + + } else if (etherType == ETHERTYPE_ARP && + (header.len >= ethHdrSize + + sizeof(struct f_arphdr))) { + struct f_arphdr *arphdr = (struct f_arphdr*)(packet + + ethHdrSize); + switch (ntohs(arphdr->arphdr.ar_op)) { + case ARPOP_REPLY: + vmaddr = arphdr->ar_sip; + break; + case ARPOP_REQUEST: + vmaddr = arphdr->ar_tip; + break; + } + } + } else if (memcmp(ether_hdr->ether_dhost, + req->macaddr, + VIR_MAC_BUFLEN) == 0) { + // packets to the VM + if (etherType == ETHERTYPE_IP && + (header.len >= ethHdrSize + + sizeof(struct iphdr))) { + struct iphdr *iphdr = (struct iphdr*)(packet + + ethHdrSize); + if ((iphdr->protocol == IPPROTO_UDP) && + (header.len >= ethHdrSize + + iphdr->ihl * 4 + + sizeof(struct udphdr))) { + struct udphdr *udphdr= (struct udphdr *) + ((char *)iphdr + iphdr->ihl * 4); + if (ntohs(udphdr->source) == 67 && + ntohs(udphdr->dest) == 68 && + header.len >= ethHdrSize + + iphdr->ihl * 4 + + sizeof(struct udphdr) + + sizeof(struct dhcp)) { + struct dhcp *dhcp = (struct dhcp *) + ((char *)udphdr + sizeof(udphdr)); + if (dhcp->op == 2 /* DHCP OFFER */) { + vmaddr = dhcp->yiaddr; + } + } + } + } + } + } + } + + done: + VIR_FREE(filter); + + if (handle) + pcap_close(handle); + + ebtablesRemoveBasicRules(NULL, req->ifname); + + if (req->status == 0) { + int ret; + char inetaddr[INET_ADDRSTRLEN]; + inet_ntop(AF_INET, &vmaddr, inetaddr, sizeof(inetaddr)); + + virNWFilterAddIpAddrForIfname(req->ifname, strdup(inetaddr)); + + ret = virNWFilterInstantiateFilterLate(NULL, + req->ifname, + req->linkdev, + req->nettype, + req->macaddr, + req->filtername, + req->filterparams, + req->driver); + VIR_DEBUG("Result from applying firewall rules on " + "%s with IP addr %s : %d\n", req->ifname, inetaddr, ret); + } + + memset(&req->thread, 0x0, sizeof(req->thread)); + + VIR_DEBUG("pcap thread terminating for interface %s\n",req->ifname); + + virNWFilterDeregisterLearnReq(req->ifname); + + virNWFilterIPAddrLearnReqFree(req); + + return 0; +} + + +/** + * virNWFilterLearnIPAddress + * @conn: pointer to virConnect object + * @ifname: the name of the interface + * @linkdev : the name of the link device; currently only used in case of a + * macvtap device + * @nettype : the type of interface + * @macaddr : the MAC address of the interface + * @filtername : the name of the top-level filter to apply to the interface + * once its IP address has been detected + * @driver : the network filter driver + * + * Instruct to learn the IP address being used on a given interface (ifname). + * Unless there already is a thread attempting to learn the IP address + * being used on the interface, a thread is started that will listen on + * the traffic being sent on the interface (or link device) with the + * MAC address that is provided. Will then launch the application of the + * firewall rules on the interface. + */ +int +virNWFilterLearnIPAddress(virConnectPtr conn, + const char *ifname, + const char *linkdev, + enum virDomainNetType nettype, + const unsigned char *macaddr, + const char *filtername, + virNWFilterHashTablePtr filterparams, + virNWFilterDriverStatePtr driver) { + int rc; + virNWFilterIPAddrLearnReqPtr req = NULL; + virNWFilterHashTablePtr ht = NULL; + + if (VIR_ALLOC(req) < 0) { + virReportOOMError(); + goto err_exit; + } + + ht = virNWFilterHashTableCreate(0); + if (ht == NULL) { + virReportOOMError(); + goto err_exit; + } + + if (virNWFilterHashTablePutAll(conn, filterparams, ht)) + goto err_exit; + + req->filtername = strdup(filtername); + if (req->filtername == NULL) { + virReportOOMError(); + goto err_exit; + } + + if (virStrcpyStatic(req->ifname, ifname) == NULL) { + virNWFilterReportError(conn, VIR_ERR_INTERNAL_ERROR, + _("Destination buffer for ifname ('%s') " + "not large enough"), ifname); + goto err_exit; + } + + if (linkdev) { + if (virStrcpyStatic(req->linkdev, linkdev) == NULL) { + virNWFilterReportError(conn, VIR_ERR_INTERNAL_ERROR, + _("Destination buffer for linkdev ('%s') " + "not large enough"), linkdev); + goto err_exit; + } + } + req->nettype = nettype; + memcpy(req->macaddr, macaddr, sizeof(req->macaddr)); + req->driver = driver; + req->filterparams = ht; + ht = NULL; + + rc = virNWFilterRegisterLearnReq(req); + + if (rc) + goto err_exit; + + if (ebtablesApplyBasicRules(conn, + ifname, + macaddr)) { + goto err_exit; + } + + if (pthread_create(&req->thread, + NULL, + learnIPAddressThread, + req) != 0) + goto err_exit; + + return 0; + +err_exit: + ebtablesRemoveBasicRules(conn, ifname); + virNWFilterIPAddrLearnReqFree(req); + virNWFilterHashTableFree(ht); + return 1; +} + +#else + +int +virNWFilterLearnIPAddress(virConnectPtr conn ATTRIBUTE_UNUSED, + const char *ifname ATTRIBUTE_UNUSED, + const char *linkdev ATTRIBUTE_UNUSED, + enum virDomainNetType nettype ATTRIBUTE_UNUSED, + const unsigned char *macaddr ATTRIBUTE_UNUSED, + const char *filtername ATTRIBUTE_UNUSED, + virNWFilterHashTablePtr filterparams ATTRIBUTE_UNUSED, + virNWFilterDriverStatePtr driver ATTRIBUTE_UNUSED) { + virNWFilterReportError(conn, VIR_ERR_INTERNAL_ERROR, + _("IP parameter must be given since libvirt " + "was not compiled with IP address learning " + "support")); + return 1; +} +#endif /* HAVE_LIBPCAP */ + + +/** + * virNWFilterLearnInit + * Initialization of this layer + */ +int +virNWFilterLearnInit(void) { + pendingLearnReq = virHashCreate(0); + if (!pendingLearnReq) { + virReportOOMError(); + return 1; + } + + if (virMutexInit(&pendingLearnReqLock)) { + virNWFilterLearnShutdown(); + return 1; + } + + ipAddressMap = virNWFilterHashTableCreate(0); + if (!ipAddressMap) { + virReportOOMError(); + virNWFilterLearnShutdown(); + return 1; + } + + if (virMutexInit(&ipAddressMapLock)) { + virNWFilterLearnShutdown(); + return 1; + } + + return 0; +} + + +/** + * virNWFilterLearnShutdown + * Shutdown of this layer + */ +void +virNWFilterLearnShutdown(void) { + virHashFree(pendingLearnReq, freeLearnReqEntry); + pendingLearnReq = NULL; + + virNWFilterHashTableFree(ipAddressMap); + ipAddressMap = NULL; +} Index: libvirt-acl/src/Makefile.am =================================================================== --- libvirt-acl.orig/src/Makefile.am +++ libvirt-acl/src/Makefile.am @@ -289,7 +289,8 @@ STORAGE_HELPER_DISK_SOURCES = \ NWFILTER_DRIVER_SOURCES = \ nwfilter/nwfilter_driver.h nwfilter/nwfilter_driver.c \ nwfilter/nwfilter_gentech_driver.c \ - nwfilter/nwfilter_ebiptables_driver.c + nwfilter/nwfilter_ebiptables_driver.c \ + nwfilter/nwfilter_learnipaddr.c # Security framework and drivers for various models SECURITY_DRIVER_SOURCES = \ @@ -740,10 +741,11 @@ else libvirt_la_LIBADD += libvirt_driver_nwfilter.la noinst_LTLIBRARIES += libvirt_driver_nwfilter.la endif -libvirt_driver_nwfilter_la_CFLAGS = \ +libvirt_driver_nwfilter_la_CFLAGS = $(LIBPCAP_CFLAGS) \ -I@top_srcdir@/src/conf +libvirt_driver_nwfilter_la_LDFLAGS = $(LIBPCAP_LIBS) if WITH_DRIVER_MODULES -libvirt_driver_nwfilter_la_LDFLAGS = -module -avoid-version ../gnulib/lib/libgnu.la +libvirt_driver_nwfilter_la_LDFLAGS += -module -avoid-version ../gnulib/lib/libgnu.la endif libvirt_driver_nwfilter_la_SOURCES = $(NWFILTER_DRIVER_SOURCES) endif @@ -887,6 +889,7 @@ libvirt_la_LDFLAGS = $(VERSION_SCRIPT_FL -version-info $(LIBVIRT_VERSION_INFO) \ $(COVERAGE_CFLAGS:-f%=-Wc,-f%) \ $(LIBXML_LIBS) \ + $(LIBPCAP_LIBS) \ $(DRIVER_MODULE_LIBS) \ $(CYGWIN_EXTRA_LDFLAGS) $(MINGW_EXTRA_LDFLAGS) libvirt_la_CFLAGS = $(COVERAGE_CFLAGS) -DIN_LIBVIRT @@ -935,8 +938,8 @@ libvirt_lxc_SOURCES = \ $(NODE_INFO_SOURCES) \ $(ENCRYPTION_CONF_SOURCES) \ $(DOMAIN_CONF_SOURCES) \ - $(NWFILTER_PARAM_CONF_SOURCES) \ - $(CPU_CONF_SOURCES) + $(CPU_CONF_SOURCES) \ + $(NWFILTER_PARAM_CONF_SOURCES) libvirt_lxc_LDFLAGS = $(WARN_CFLAGS) $(COVERAGE_LDCFLAGS) $(CAPNG_LIBS) $(YAJL_LIBS) libvirt_lxc_LDADD = $(LIBXML_LIBS) $(NUMACTL_LIBS) ../gnulib/lib/libgnu.la libvirt_lxc_CFLAGS = \ Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c @@ -24,6 +24,9 @@ #include <config.h> #include <stdint.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <linux/if.h> #include "internal.h" @@ -34,12 +37,16 @@ #include "virterror_internal.h" #include "nwfilter_gentech_driver.h" #include "nwfilter_ebiptables_driver.h" +#include "nwfilter_learnipaddr.h" #define VIR_FROM_THIS VIR_FROM_NWFILTER #define NWFILTER_STD_VAR_MAC "MAC" +#define NWFILTER_STD_VAR_IP "IP" + +static int _virNWFilterTeardownFilter(const char *ifname); static virNWFilterTechDriverPtr filter_tech_drivers[] = { @@ -111,6 +118,8 @@ virNWFilterRuleInstFree(virNWFilterRuleI * @tables: pointer to hash tabel to add values to * @macaddr: The string of the MAC address to add to the hash table, * may be NULL + * @ipaddr: The string of the IP address to add to the hash table; + * may be NULL * * Returns 0 in case of success, 1 in case an error happened with * error having been reported. @@ -120,7 +129,8 @@ virNWFilterRuleInstFree(virNWFilterRuleI static int virNWFilterVarHashmapAddStdValues(virConnectPtr conn, virNWFilterHashTablePtr table, - char *macaddr) + char *macaddr, + char *ipaddr) { if (macaddr) { if (virHashAddEntry(table->hashTable, @@ -132,6 +142,16 @@ virNWFilterVarHashmapAddStdValues(virCon } } + if (ipaddr) { + if (virHashAddEntry(table->hashTable, + NWFILTER_STD_VAR_IP, + ipaddr) < 0) { + virNWFilterReportError(conn, VIR_ERR_INTERNAL_ERROR, + "%s", _("Could not add variable 'IP' to hashmap")); + return 1; + } + } + return 0; } @@ -140,23 +160,26 @@ virNWFilterVarHashmapAddStdValues(virCon * virNWFilterCreateVarHashmap: * @conn: pointer to virConnect object * @macaddr: pointer to string containing formatted MAC address of interface + * @ipaddr: pointer to string containing formatted IP address used by + * VM on this interface; may be NULL * * Create a hashmap used for evaluating the firewall rules. Initializes - * it with the standard variable 'MAC'. + * it with the standard variable 'MAC' and 'IP' if provided. * * Returns pointer to hashmap, NULL if an error occcurred and error message * is attached to the virConnect object. */ virNWFilterHashTablePtr virNWFilterCreateVarHashmap(virConnectPtr conn, - char *macaddr) { + char *macaddr, + char *ipaddr) { virNWFilterHashTablePtr table = virNWFilterHashTableCreate(0); if (!table) { virReportOOMError(); return NULL; } - if (virNWFilterVarHashmapAddStdValues(conn, table, macaddr)) { + if (virNWFilterVarHashmapAddStdValues(conn, table, macaddr, ipaddr)) { virNWFilterHashTableFree(table); return NULL; } @@ -284,9 +307,9 @@ _virNWFilterInstantiateRec(virConnectPtr virNWFilterHashTablePtr vars, int *nEntries, virNWFilterRuleInstPtr **insts, - enum instCase useNewFilter, int *foundNewFilter) + enum instCase useNewFilter, int *foundNewFilter, + virNWFilterDriverStatePtr driver) { - virNWFilterDriverStatePtr driver = conn->nwfilterPrivateData; virNWFilterPoolObjPtr obj; int rc = 0; int i; @@ -365,7 +388,96 @@ _virNWFilterInstantiateRec(virConnectPtr tmpvars, nEntries, insts, useNewFilter, - foundNewFilter); + foundNewFilter, + driver); + + virNWFilterHashTableFree(tmpvars); + + virNWFilterPoolObjUnlock(obj); + if (rc) + break; + } else { + virNWFilterReportError(conn, VIR_ERR_INTERNAL_ERROR, + _("referenced filter '%s' is missing"), + inc->filterref); + rc = 1; + break; + } + } + } + return rc; +} + + +static int +virNWFilterDetermineMissingVarsRec(virConnectPtr conn, + virNWFilterDefPtr filter, + virNWFilterHashTablePtr vars, + virNWFilterHashTablePtr missing_vars, + int useNewFilter, + virNWFilterDriverStatePtr driver) +{ + virNWFilterPoolObjPtr obj; + int rc = 0; + int i, j; + virNWFilterDefPtr next_filter; + + for (i = 0; i < filter->nentries; i++) { + virNWFilterRuleDefPtr rule = filter->filterEntries[i]->rule; + virNWFilterIncludeDefPtr inc = filter->filterEntries[i]->include; + if (rule) { + // check all variables of this rule + for (j = 0; j < rule->nvars; j++) { + if (!virHashLookup(vars->hashTable, rule->vars[j])) { + virNWFilterHashTablePut(missing_vars, rule->vars[j], + strdup("1"), 1); + } + } + } else if (inc) { + VIR_DEBUG("Following filter %s\n", inc->filterref); + obj = virNWFilterPoolObjFindByName(&driver->pools, + inc->filterref); + if (obj) { + + if (obj->wantRemoved) { + virNWFilterReportError(conn, VIR_ERR_NO_NWFILTER, + _("Filter '%s' is in use."), + inc->filterref); + rc = 1; + virNWFilterPoolObjUnlock(obj); + break; + } + + // create a temporary hashmap for depth-first tree traversal + virNWFilterHashTablePtr tmpvars = + virNWFilterCreateVarsFrom(conn, + inc->params, + vars); + if (!tmpvars) { + virReportOOMError(); + rc = 1; + virNWFilterPoolObjUnlock(obj); + break; + } + + next_filter = obj->def; + + switch (useNewFilter) { + case INSTANTIATE_FOLLOW_NEWFILTER: + if (obj->newDef) { + next_filter = obj->newDef; + } + break; + case INSTANTIATE_ALWAYS: + break; + } + + rc = virNWFilterDetermineMissingVarsRec(conn, + next_filter, + tmpvars, + missing_vars, + useNewFilter, + driver); virNWFilterHashTableFree(tmpvars); @@ -438,9 +550,12 @@ virNWFilterInstantiate(virConnectPtr con enum virDomainNetType nettype, virNWFilterDefPtr filter, const char *ifname, + const char *linkdev, virNWFilterHashTablePtr vars, enum instCase useNewFilter, int *foundNewFilter, - bool teardownOld) + bool teardownOld, + const unsigned char *macaddr, + virNWFilterDriverStatePtr driver) { int rc; int j, nptrs; @@ -449,6 +564,44 @@ virNWFilterInstantiate(virConnectPtr con void **ptrs = NULL; int instantiate = 1; + virNWFilterLockFilterUpdates(); + + virNWFilterHashTablePtr missing_vars = virNWFilterHashTableCreate(0); + if (!missing_vars) { + virReportOOMError(); + rc = 1; + goto err_exit; + } + + rc = virNWFilterDetermineMissingVarsRec(conn, + filter, + vars, + missing_vars, + useNewFilter, + driver); + if (rc) + goto err_exit; + + if (virHashSize(missing_vars->hashTable) == 1) { + if (virHashLookup(missing_vars->hashTable, + NWFILTER_STD_VAR_IP) != NULL) { + if (virNWFilterLookupLearnReq(ifname) == NULL) { + rc = virNWFilterLearnIPAddress(conn, + ifname, + linkdev, + nettype, macaddr, + filter->name, + vars, driver); + } + goto err_exit; + } + rc = 1; + goto err_exit; + } else if (virHashSize(missing_vars->hashTable) > 1) { + rc = 1; + goto err_exit; + } + rc = _virNWFilterInstantiateRec(conn, techdriver, nettype, @@ -456,7 +609,8 @@ virNWFilterInstantiate(virConnectPtr con ifname, vars, &nEntries, &insts, - useNewFilter, foundNewFilter); + useNewFilter, foundNewFilter, + driver); if (rc) goto err_exit; @@ -487,24 +641,33 @@ virNWFilterInstantiate(virConnectPtr con err_exit: + virNWFilterUnlockFilterUpdates(); + for (j = 0; j < nEntries; j++) virNWFilterRuleInstFree(insts[j]); VIR_FREE(insts); + virNWFilterHashTableFree(missing_vars); + return rc; } static int -_virNWFilterInstantiateFilter(virConnectPtr conn, - const virDomainNetDefPtr net, - bool teardownOld, - enum instCase useNewFilter) +__virNWFilterInstantiateFilter(virConnectPtr conn, + bool teardownOld, + const char *ifname, + const char *linkdev, + enum virDomainNetType nettype, + const unsigned char *macaddr, + const char *filtername, + virNWFilterHashTablePtr filterparams, + enum instCase useNewFilter, + virNWFilterDriverStatePtr driver) { int rc; const char *drvname = EBIPTABLES_DRIVER_ID; - virNWFilterDriverStatePtr driver = conn->nwfilterPrivateData; virNWFilterTechDriverPtr techdriver; virNWFilterPoolObjPtr obj; virNWFilterHashTablePtr vars, vars1; @@ -512,6 +675,8 @@ _virNWFilterInstantiateFilter(virConnect char vmmacaddr[VIR_MAC_STRING_BUFLEN] = {0}; int foundNewFilter = 0; char *str_macaddr = NULL; + const char *ipaddr; + char *str_ipaddr = NULL; techdriver = virNWFilterTechDriverForName(drvname); @@ -523,25 +688,25 @@ _virNWFilterInstantiateFilter(virConnect return 1; } - VIR_DEBUG("filter name: %s\n", net->filter); + VIR_DEBUG("filter name: %s\n", filtername); - obj = virNWFilterPoolObjFindByName(&driver->pools, net->filter); + obj = virNWFilterPoolObjFindByName(&driver->pools, filtername); if (!obj) { virNWFilterReportError(conn, VIR_ERR_NO_NWFILTER, _("Could not find filter '%s'"), - net->filter); + filtername); return 1; } if (obj->wantRemoved) { virNWFilterReportError(conn, VIR_ERR_NO_NWFILTER, _("Filter '%s' is in use."), - net->filter); + filtername); rc = 1; goto err_exit; } - virFormatMacAddr(net->mac, vmmacaddr); + virFormatMacAddr(macaddr, vmmacaddr); str_macaddr = strdup(vmmacaddr); if (!str_macaddr) { virReportOOMError(); @@ -549,18 +714,29 @@ _virNWFilterInstantiateFilter(virConnect goto err_exit; } + ipaddr = virNWFilterGetIpAddrForIfname(ifname); + if (ipaddr) { + str_ipaddr = strdup(ipaddr); + if (!str_ipaddr) { + virReportOOMError(); + rc = 1; + goto err_exit; + } + } + vars1 = virNWFilterCreateVarHashmap(conn, - str_macaddr); + str_macaddr, str_ipaddr); if (!vars1) { rc = 1; goto err_exit; } str_macaddr = NULL; + str_ipaddr = NULL; vars = virNWFilterCreateVarsFrom(conn, vars1, - net->filterparams); + filterparams); if (!vars) { rc = 1; goto err_exit_vars1; @@ -582,12 +758,15 @@ _virNWFilterInstantiateFilter(virConnect rc = virNWFilterInstantiate(conn, techdriver, - net->type, + nettype, filter, - net->ifname, + ifname, + linkdev, vars, useNewFilter, &foundNewFilter, - teardownOld); + teardownOld, + macaddr, + driver); virNWFilterHashTableFree(vars); @@ -595,15 +774,180 @@ err_exit_vars1: virNWFilterHashTableFree(vars1); err_exit: - virNWFilterPoolObjUnlock(obj); + VIR_FREE(str_ipaddr); VIR_FREE(str_macaddr); return rc; } +static int +_virNWFilterInstantiateFilter(virConnectPtr conn, + const virDomainNetDefPtr net, + bool teardownOld, + enum instCase useNewFilter) +{ + const char *linkdev = (net->type == VIR_DOMAIN_NET_TYPE_DIRECT) + ? net->data.direct.linkdev + : NULL; + return __virNWFilterInstantiateFilter(conn, + teardownOld, + net->ifname, + linkdev, + net->type, + net->mac, + net->filter, + net->filterparams, + useNewFilter, + conn->nwfilterPrivateData); +} + + +// FIXME: move chgIfFlags, ifUp, checkIf into common file & share w/ macvtap.c + +/* + * chgIfFlags: Change flags on an interface + * @ifname : name of the interface + * @flagclear : the flags to clear + * @flagset : the flags to set + * + * The new flags of the interface will be calculated as + * flagmask = (~0 ^ flagclear) + * newflags = (curflags & flagmask) | flagset; + * + * Returns 0 on success, errno on failure. + */ +static int chgIfFlags(const char *ifname, short flagclear, short flagset) { + struct ifreq ifr; + int rc = 0; + int flags; + short flagmask = (~0 ^ flagclear); + int fd = socket(PF_PACKET, SOCK_DGRAM, 0); + + if (fd < 0) + return errno; + + if (virStrncpy(ifr.ifr_name, + ifname, strlen(ifname), sizeof(ifr.ifr_name)) == NULL) { + rc = ENODEV; + goto err_exit; + } + + if (ioctl(fd, SIOCGIFFLAGS, &ifr) < 0) { + rc = errno; + goto err_exit; + } + + flags = (ifr.ifr_flags & flagmask) | flagset; + + if (ifr.ifr_flags != flags) { + ifr.ifr_flags = flags; + + if (ioctl(fd, SIOCSIFFLAGS, &ifr) < 0) + rc = errno; + } + +err_exit: + close(fd); + return rc; +} + +/* + * ifUp + * @name: name of the interface + * @up: 1 for up, 0 for down + * + * Function to control if an interface is activated (up, 1) or not (down, 0) + * + * Returns 0 in case of success or an errno code in case of failure. + */ +static int +ifUp(const char *name, int up) +{ + return chgIfFlags(name, + (up) ? 0 : IFF_UP, + (up) ? IFF_UP : 0); +} + + +/** + * checkIf + * + * @ifname: Name of the interface + * @macaddr: expected MAC address of the interface + * + * FIXME: the interface's index is another good parameter to check + * + * Determine whether a given interface is still available. If so, + * it must have the given MAC address. + * + * Returns an error code ENODEV in case the interface does not exist + * anymore or its MAC address is different, 0 otherwise. + */ +int +checkIf(const char *ifname, const unsigned char *macaddr) +{ + struct ifreq ifr; + int fd = socket(PF_PACKET, SOCK_DGRAM, 0); + int rc = 0; + + if (fd < 0) + return errno; + + if (virStrncpy(ifr.ifr_name, + ifname, strlen(ifname), sizeof(ifr.ifr_name)) == NULL) { + rc = ENODEV; + goto err_exit; + } + + if (ioctl(fd, SIOCGIFHWADDR, &ifr) < 0) { + rc = errno; + goto err_exit; + } + + if (memcmp(&ifr.ifr_hwaddr.sa_data, macaddr, 6) != 0) + rc = ENODEV; + + err_exit: + close(fd); + return rc; +} + + +int +virNWFilterInstantiateFilterLate(virConnectPtr conn, + const char *ifname, + const char *linkdev, + enum virDomainNetType nettype, + const unsigned char *macaddr, + const char *filtername, + virNWFilterHashTablePtr filterparams, + virNWFilterDriverStatePtr driver) +{ + int rc; + rc = __virNWFilterInstantiateFilter(conn, + 1, + ifname, + linkdev, + nettype, + macaddr, + filtername, + filterparams, + INSTANTIATE_ALWAYS, + driver); + if (rc) { + //something went wrong... 'DOWN' the interface + if (ifUp(ifname ,0)) { + // assuming interface disappeared... + _virNWFilterTeardownFilter(ifname); + } + } + return rc; +} + + int virNWFilterInstantiateFilter(virConnectPtr conn, const virDomainNetDefPtr net) @@ -660,8 +1004,8 @@ virNWFilterTearOldFilter(virConnectPtr c } -int -virNWFilterTeardownFilter(const virDomainNetDefPtr net) +static int +_virNWFilterTeardownFilter(const char *ifname) { const char *drvname = EBIPTABLES_DRIVER_ID; virNWFilterTechDriverPtr techdriver; @@ -676,13 +1020,21 @@ virNWFilterTeardownFilter(const virDomai #endif return 1; } + techdriver->allTeardown(ifname); - techdriver->allTeardown(net->ifname); + virNWFilterDelIpAddrForIfname(ifname); return 0; } +int +virNWFilterTeardownFilter(const virDomainNetDefPtr net) +{ + return _virNWFilterTeardownFilter(net->ifname); +} + + void virNWFilterDomainFWUpdateCB(void *payload, const char *name ATTRIBUTE_UNUSED, Index: libvirt-acl/src/nwfilter/nwfilter_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_driver.c @@ -37,6 +37,8 @@ #include "nwfilter_gentech_driver.h" +#include "nwfilter_learnipaddr.h" + #define VIR_FROM_THIS VIR_FROM_NWFILTER #define nwfilterLog(msg...) fprintf(stderr, msg) @@ -65,9 +67,12 @@ static int nwfilterDriverStartup(int privileged) { char *base = NULL; - if (virNWFilterConfLayerInit(virNWFilterDomainFWUpdateCB) < 0) + if (virNWFilterLearnInit() < 0) return -1; + if (virNWFilterConfLayerInit(virNWFilterDomainFWUpdateCB) < 0) + goto conf_init_err; + if (VIR_ALLOC(driverState) < 0) goto alloc_err_exit; @@ -120,6 +125,9 @@ error: alloc_err_exit: virNWFilterConfLayerShutdown(); +conf_init_err: + virNWFilterLearnShutdown(); + return -1; } @@ -413,5 +421,6 @@ static virStateDriver stateDriver = { int nwfilterRegister(void) { virRegisterNWFilterDriver(&nwfilterDriver); virRegisterStateDriver(&stateDriver); + virNWFilterLearnInit(); return 0; } Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.h =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.h +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.h @@ -46,13 +46,25 @@ int virNWFilterRollbackUpdateFilter(virC int virNWFilterTearOldFilter(virConnectPtr conn, const virDomainNetDefPtr net); +int virNWFilterInstantiateFilterLate(virConnectPtr conn, + const char *ifname, + const char *linkdev, + enum virDomainNetType nettype, + const unsigned char *macaddr, + const char *filtername, + virNWFilterHashTablePtr filterparams, + virNWFilterDriverStatePtr driver); + int virNWFilterTeardownFilter(const virDomainNetDefPtr net); virNWFilterHashTablePtr virNWFilterCreateVarHashmap(virConnectPtr conn, - char *macaddr); + char *macaddr, + char *ipaddr); void virNWFilterDomainFWUpdateCB(void *payload, const char *name ATTRIBUTE_UNUSED, void *data); +int checkIf(const char *ifname, const unsigned char *macaddr); + #endif Index: libvirt-acl/src/libvirt_private.syms =================================================================== --- libvirt-acl.orig/src/libvirt_private.syms +++ libvirt-acl/src/libvirt_private.syms @@ -488,6 +488,8 @@ virNWFilterRegisterCallbackDriver; virNWFilterTestUnassignDef; virNWFilterConfLayerInit; virNWFilterConfLayerShutdown; +virNWFilterLockFilterUpdates; +virNWFilterUnlockFilterUpdates; #nwfilter_params.h @@ -503,6 +505,16 @@ virNWFilterInstantiateFilter; virNWFilterTeardownFilter; +#nwfilter_learnipaddr.h +ipAddressMap; +ipAddressMapLock; +pendingLearnReq; +pendingLearnReqLock; +virNWFilterGetIpAddrForIfname; +virNWFilterDelIpAddrForIfname; +virNWFilterLookupLearnReq; + + # pci.h pciGetDevice; pciFreeDevice; Index: libvirt-acl/src/conf/nwfilter_conf.c =================================================================== --- libvirt-acl.orig/src/conf/nwfilter_conf.c +++ libvirt-acl/src/conf/nwfilter_conf.c @@ -111,17 +111,18 @@ struct int_map { */ static virMutex updateMutex; -static void +void virNWFilterLockFilterUpdates(void) { virMutexLock(&updateMutex); } -static void +void virNWFilterUnlockFilterUpdates(void) { virMutexUnlock(&updateMutex); } + /* * attribute names for the rules XML */ @@ -2731,7 +2732,7 @@ int virNWFilterConfLayerInit(virHashIter { virNWFilterDomainFWUpdateCB = domUpdateCB; - if (virMutexInit(&updateMutex)) + if (virMutexInitRecursive(&updateMutex)) return 1; if (virNWFilterParamConfLayerInit()) Index: libvirt-acl/src/conf/nwfilter_conf.h =================================================================== --- libvirt-acl.orig/src/conf/nwfilter_conf.h +++ libvirt-acl/src/conf/nwfilter_conf.h @@ -565,6 +565,9 @@ virNWFilterDefPtr virNWFilterDefParseFil void virNWFilterPoolObjLock(virNWFilterPoolObjPtr obj); void virNWFilterPoolObjUnlock(virNWFilterPoolObjPtr obj); +void virNWFilterLockFilterUpdates(void); +void virNWFilterUnlockFilterUpdates(void); + int virNWFilterConfLayerInit(virHashIterator domUpdateCB); void virNWFilterConfLayerShutdown(void); Index: libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_ebiptables_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.c @@ -2533,6 +2533,106 @@ ebiptablesInstCommand(virConnectPtr conn } +int +ebtablesApplyBasicRules(virConnectPtr conn, + const char *ifname, + const unsigned char *macaddr) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + int cli_status; + char chain[MAX_CHAINNAME_LENGTH]; + char chainPrefix = CHAINPREFIX_HOST_IN_TEMP; + char macaddr_str[VIR_MAC_STRING_BUFLEN]; + + virFormatMacAddr(macaddr, macaddr_str); + + ebtablesUnlinkTmpRootChain(conn, &buf, 1, ifname); + ebtablesUnlinkTmpRootChain(conn, &buf, 0, ifname); + ebtablesRemoveTmpSubChains(conn, &buf, ifname); + ebtablesRemoveTmpRootChain(conn, &buf, 1, ifname); + ebtablesRemoveTmpRootChain(conn, &buf, 0, ifname); + ebiptablesExecCLI(conn, &buf, &cli_status); + + ebtablesCreateTmpRootChain(conn, &buf, 1, ifname, 1); + + PRINT_ROOT_CHAIN(chain, chainPrefix, ifname); + virBufferVSprintf(&buf, + CMD_DEF(EBTABLES_CMD + " -t %s -A %s -s ! %s -j DROP") CMD_SEPARATOR + CMD_EXEC + "%s", + + EBTABLES_DEFAULT_TABLE, + chain, + macaddr_str, + CMD_STOPONERR(1)); + + virBufferVSprintf(&buf, + CMD_DEF(EBTABLES_CMD + " -t %s -A %s -p IPv4 -j ACCEPT") CMD_SEPARATOR + CMD_EXEC + "%s", + + EBTABLES_DEFAULT_TABLE, + chain, + CMD_STOPONERR(1)); + + virBufferVSprintf(&buf, + CMD_DEF(EBTABLES_CMD + " -t %s -A %s -p ARP -j ACCEPT") CMD_SEPARATOR + CMD_EXEC + "%s", + + EBTABLES_DEFAULT_TABLE, + chain, + CMD_STOPONERR(1)); + + virBufferVSprintf(&buf, + CMD_DEF(EBTABLES_CMD + " -t %s -A %s -j DROP") CMD_SEPARATOR + CMD_EXEC + "%s", + + EBTABLES_DEFAULT_TABLE, + chain, + CMD_STOPONERR(1)); + + ebtablesLinkTmpRootChain(conn, &buf, 1, ifname, 1); + + if (ebiptablesExecCLI(conn, &buf, &cli_status) || cli_status != 0) + goto tear_down_tmpebchains; + + return 0; + +tear_down_tmpebchains: + ebtablesRemoveBasicRules(conn, ifname); + + virNWFilterReportError(conn, VIR_ERR_BUILD_FIREWALL, + "%s", + _("Some rules could not be created.")); + + return 1; +} + + +int +ebtablesRemoveBasicRules(virConnectPtr conn, + const char *ifname) +{ + virBuffer buf = VIR_BUFFER_INITIALIZER; + int cli_status; + + ebtablesUnlinkTmpRootChain(conn, &buf, 1, ifname); + ebtablesUnlinkTmpRootChain(conn, &buf, 0, ifname); + ebtablesRemoveTmpSubChains(conn, &buf, ifname); + ebtablesRemoveTmpRootChain(conn, &buf, 1, ifname); + ebtablesRemoveTmpRootChain(conn, &buf, 0, ifname); + + ebiptablesExecCLI(conn, &buf, &cli_status); + return 0; +} + + static int ebiptablesRuleOrderSort(const void *a, const void *b) { Index: libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.h =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_ebiptables_driver.h +++ libvirt-acl/src/nwfilter/nwfilter_ebiptables_driver.h @@ -45,4 +45,11 @@ extern virNWFilterTechDriver ebiptables_ # define EBIPTABLES_DRIVER_ID "ebiptables" + +int ebtablesApplyBasicRules(virConnectPtr conn, + const char *ifname, + const unsigned char *macaddr); +int ebtablesRemoveBasicRules(virConnectPtr conn, + const char *ifname); + #endif Index: libvirt-acl/configure.ac =================================================================== --- libvirt-acl.orig/configure.ac +++ libvirt-acl/configure.ac @@ -41,6 +41,7 @@ XMLRPC_REQUIRED=1.14.0 HAL_REQUIRED=0.5.0 DEVMAPPER_REQUIRED=1.0.0 LIBCURL_REQUIRED="7.18.0" +LIBPCAP_REQUIRED="1.0.0" dnl Checks for C compiler. AC_PROG_CC @@ -1045,6 +1046,39 @@ AC_SUBST([NUMACTL_CFLAGS]) AC_SUBST([NUMACTL_LIBS]) +dnl pcap lib +LIBPCAP_CONFIG="pcap-config" +LIBPCAP_CFLAGS="" +LIBPCAP_LIBS="" +LIBPCAP_FOUND="no" + +AC_ARG_WITH([libpcap], AC_HELP_STRING([--with-libpcap=@<:@PFX@:>@], [libpcap location])) +if test "$with_qemu" = "yes"; then + if test "x$with_libpcap" != "xno" ; then + if test "x$with_libpcap" != "x" ; then + LIBPCAP_CONFIG=$with_libpcap/bin/$LIBPCAP_CONFIG + fi + AC_MSG_CHECKING(libpcap $LIBPCAP_CONFIG >= $LIBPCAP_REQUIRED ) + if ! $LIBPCAP_CONFIG --libs > /dev/null 2>&1 ; then + AC_MSG_RESULT(no) + else + LIBPCAP_LIBS="`$LIBPCAP_CONFIG --libs`" + LIBPCAP_CFLAGS="`$LIBPCAP_CONFIG --cflags`" + LIBPCAP_FOUND="yes" + AC_MSG_RESULT(yes) + fi + fi +fi + +if test "x$LIBPCAP_FOUND" = "xyes"; then + AC_DEFINE_UNQUOTED([HAVE_LIBPCAP], 1, [whether libpcap can be used]) +fi + +AC_SUBST([LIBPCAP_CFLAGS]) +AC_SUBST([LIBPCAP_LIBS]) + + + dnl dnl Checks for the UML driver dnl @@ -2129,6 +2163,11 @@ AC_MSG_NOTICE([ xmlrpc: $XMLRPC_CFLAGS else AC_MSG_NOTICE([ xmlrpc: no]) fi +if test "$with_qemu" = "yes" ; then +AC_MSG_NOTICE([ pcap: $LIBPCAP_CFLAGS $LIBPCAP_LIBS]) +else +AC_MSG_NOTICE([ pcap: no]) +fi AC_MSG_NOTICE([]) AC_MSG_NOTICE([Test suite]) AC_MSG_NOTICE([])

On 03/30/2010 11:56 AM, Stefan Berger wrote:
Subject: Support for learning a VM's IP address
This patch implements support for learning a VM's IP address. It uses the pcap library to listen on the VM's backend network interface (tap) or the physical ethernet device (macvtap) and tries to capture packets with source or destination MAC address of the VM and learn from DHCP Offers, ARP traffic, or first-sent IPv4 packet what the IP address of the VM's interface is. This then allows to instantiate the network traffic filtering rules without the user having to provide the IP parameter somewhere in the filter description or in the interface description as a parameter. This only supports to detect the parameter IP, which is for the assumed single IPv4 address of a VM. There is not support for interfaces that may have multiple IP addresses (IP aliasing) or IPv6 that may then require more than one valid IP address to be detected. A VM can have multiple independent interfaces that each uses a different IP address and in that case it will be attempted to detect each one of the address independently.
Sounds interesting.
--- configure.ac | 39 ++
Should there also be an adjustment to the .rpm spec files, to mention that we now might depend on libpcap (and building of all features depends on libpcap-devel)?
Index: libvirt-acl/src/nwfilter/nwfilter_learnipaddr.c ... + +#ifdef HAVE_LIBPCAP +#include <pcap.h> +#endif
Fails 'make syntax-check' if you install cppi (which is now part of Fedora).
+ +#include <fcntl.h> +#include <sys/ioctl.h> + +#include <arpa/inet.h> +#include <net/ethernet.h> +#include <netinet/ip.h> +#include <netinet/udp.h> +#include <net/if_arp.h> +#include <linux/if.h>
Can we guarantee that this header always exists, or should it be conditionally included?
+ + +/* structure of an ARP request/reply message */ +struct f_arphdr { + struct arphdr arphdr; + uint8_t ar_sha[ETH_ALEN]; + uint32_t ar_sip; + uint8_t ar_tha[ETH_ALEN]; + uint32_t ar_tip; +} __attribute__((packed));
This assumes GCC; do we ever intend to support other compilers?
+/** + * virNWFilterLearnInit + * Initialization of this layer + */ +int +virNWFilterLearnInit(void) { + pendingLearnReq = virHashCreate(0); + if (!pendingLearnReq) { + virReportOOMError(); + return 1; + } + + if (virMutexInit(&pendingLearnReqLock)) { + virNWFilterLearnShutdown(); + return 1; + }
Is this function leaking memory if it fails mid-way through? Or are you requiring that the caller will call virNWFilterShutdown even if virNWFilterLearnInit returns failure?
Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c @@ -24,6 +24,9 @@ #include <config.h>
#include <stdint.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <linux/if.h>
Another instance of using a non-portable header. I've read through the patch, and didn't spot many glaring errors. I have not tested it, though, and would rather defer to others to also look through it before approving it, since it is rather large. -- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org

Eric Blake <eblake@redhat.com> wrote on 03/31/2010 02:28:09 PM:
On 03/30/2010 11:56 AM, Stefan Berger wrote:
Subject: Support for learning a VM's IP address
This patch implements support for learning a VM's IP address. It uses the pcap library to listen on the VM's backend network interface (tap) or the physical ethernet device (macvtap) and tries to capture packets with source or destination MAC address of the VM and learn from DHCP Offers, ARP traffic, or first-sent IPv4 packet what the IP address of the VM's interface is. This then allows to instantiate the network traffic filtering rules without the user having to provide the IP parameter somewhere in the filter description or in the interface description as a parameter. This only supports to detect the parameter IP, which is for the assumed single IPv4 address of a VM. There is not support for interfaces that may have multiple IP addresses (IP aliasing) or IPv6 that may then require more than one valid IP address to be detected. A VM can have multiple independent interfaces that
each
uses a different IP address and in that case it will be attempted to detect each one of the address independently.
Sounds interesting.
--- configure.ac | 39 ++
Should there also be an adjustment to the .rpm spec files, to mention that we now might depend on libpcap (and building of all features depends on libpcap-devel)?
Correct.
Index: libvirt-acl/src/nwfilter/nwfilter_learnipaddr.c ... + +#ifdef HAVE_LIBPCAP +#include <pcap.h> +#endif
Fails 'make syntax-check' if you install cppi (which is now part of
Fedora). I don't have that install. I know, indentation after the '#'.
+ +#include <fcntl.h> +#include <sys/ioctl.h> + +#include <arpa/inet.h> +#include <net/ethernet.h> +#include <netinet/ip.h> +#include <netinet/udp.h> +#include <net/if_arp.h> +#include <linux/if.h>
Can we guarantee that this header always exists, or should it be conditionally included?
I guess this would only work on Linux, not on OpenSolaris or mingw. So, yes, conditionally including is probably the right thing. What would be the right condition to use?
+ + +/* structure of an ARP request/reply message */ +struct f_arphdr { + struct arphdr arphdr; + uint8_t ar_sha[ETH_ALEN]; + uint32_t ar_sip; + uint8_t ar_tha[ETH_ALEN]; + uint32_t ar_tip; +} __attribute__((packed));
This assumes GCC; do we ever intend to support other compilers?
Hm, it should be #defined in internal.h, I guess that would solve part of the problem. Are other compilers already being used ?
+/** + * virNWFilterLearnInit + * Initialization of this layer + */ +int +virNWFilterLearnInit(void) { + pendingLearnReq = virHashCreate(0); + if (!pendingLearnReq) { + virReportOOMError(); + return 1; + } + + if (virMutexInit(&pendingLearnReqLock)) { + virNWFilterLearnShutdown(); + return 1; + }
Is this function leaking memory if it fails mid-way through? Or are you requiring that the caller will call virNWFilterShutdown even if virNWFilterLearnInit returns failure?
No, it shouldn't. Rather than cleaning up the objects that were created in this function before it failed I call into the shutdown function to free them.
Index: libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c =================================================================== --- libvirt-acl.orig/src/nwfilter/nwfilter_gentech_driver.c +++ libvirt-acl/src/nwfilter/nwfilter_gentech_driver.c @@ -24,6 +24,9 @@ #include <config.h>
#include <stdint.h> +#include <sys/socket.h> +#include <sys/ioctl.h> +#include <linux/if.h>
Another instance of using a non-portable header.
Same issue, yes.
I've read through the patch, and didn't spot many glaring errors. I have not tested it, though, and would rather defer to others to also look through it before approving it, since it is rather large.
Ok. Stefan
-- Eric Blake eblake@redhat.com +1-801-349-2682 Libvirt virtualization library http://libvirt.org
[attachment "signature.asc" deleted by Stefan Berger/Watson/IBM]

On Tue, Mar 30, 2010 at 01:56:52PM -0400, Stefan Berger wrote:
Subject: Support for learning a VM's IP address
A caveat: The algorithm does not know which one is the appropriate IP address of a VM. If the VM spoofs an IP address in its first ARP traffic or IPv4 packets its filtering rules will be instantiated for this IP address, thus 'locking' it to the found IP address. So, it's still 'safer' to explicitly provide the IP address of a VM's interface in the filter description if it is known beforehand.
While this code is very clever, I'm not really convinced that having a learning capability that snifs arbitrary IP packets for an address is desirable. The primary task of the nwfilter mechanism is to provide secure isolation of the VM from other VMs & network protocols. Basing this ontop of a learning mode that we know can be trivially poisoned/exploited by sending fake ARPs just doesn't seem like a good plan - it is providing users a false sense of security. The only way I could see this working in a reasonably secure manner is to start from the assigned MAC address that we know & can trust. Then listen for DHCP OFFERS (IPv4/6) matching the MAC address, and extract the IP from that. This assumes DHCP OFFERS come from a trusted server, so we need to make sure that other VMs can't spoof DHCP OFFERS. Either the admin could include a DHCP blocking rule in the nwfilter config for all VMs (needs to be on all hosts), or have a host config parameter for nwfilter to specify the trusted DHCP server address. If done right, this gives a IP addr learning mode which the VM can't poison with an IP of its choosing. For IPv6 the network might use DHCPv6 which we can procss in much the same way, or it might be doing stateless autoconfig. So we'd need some host level config parameter to specify whether to learn based on DHCPv6, or based on the router advertisments. In the latter case a VM auto-assigns itself an IPv6 address based on the router prefix + its MAC address. So if we spot the router prefix + know the MAC addr we can safely set the IPv6 addr filter Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Please respond to "Daniel P. Berrange"
On Tue, Mar 30, 2010 at 01:56:52PM -0400, Stefan Berger wrote:
Subject: Support for learning a VM's IP address
A caveat: The algorithm does not know which one is the appropriate IP address of a VM. If the VM spoofs an IP address in its first ARP
or IPv4 packets its filtering rules will be instantiated for this IP address, thus 'locking' it to the found IP address. So, it's still 'safer' to explicitly provide the IP address of a VM's interface in
"Daniel P. Berrange" <berrange@redhat.com> wrote on 03/31/2010 03:44:48 PM: traffic the
filter description if it is known beforehand.
While this code is very clever, I'm not really convinced that having a learning capability that snifs arbitrary IP packets for an address is desirable. The primary task of the nwfilter mechanism is to provide secure isolation of the VM from other VMs & network protocols. Basing this ontop of a learning mode that we know can be trivially poisoned/exploited by sending fake ARPs just doesn't seem like a good plan - it is providing users a false sense of security.
That's correct. It mostly provides 'convenience'.
The only way I could see this working in a reasonably secure manner is
start from the assigned MAC address that we know & can trust. Then
DHCP OFFERS (IPv4/6) matching the MAC address, and extract the IP from
This assumes DHCP OFFERS come from a trusted server, so we need to make sure that other VMs can't spoof DHCP OFFERS. Either the admin could include a DHCP blocking rule in the nwfilter config for all VMs (needs to be on all hosts), or have a host config parameter for nwfilter to specify the
to listen for that. I am actually doing that, though I am only comparing the destination MAC address in that case so far and not the MAC address in the DHCP OFFER itself. The DHCP response is unicasted, so I don't currently compare the MAC address in the DHCP OFFER, but that would be trivial to add. The problem with libpcap is that the sockets it is using don't give a guarantee that none of the packets will be missed -- afaik. Missing that one DHCP OFFER could then be fatal and waiting for the lease timeout probably not an option. Are there other options? I'd like to keep the algorithm as much as possible as it is now but be able to tell the thread to only look for dhcp offers for example, provided that this could be done in a reliable manner. If a trusted DHCP server was provided, the thread could be acting this way, otherwise it could rely on ARP, IPv4 or DHCP Offer. It would be a matter of configuration of libvirt how the learning actually works. trusted
DHCP server address. If done right, this gives a IP addr learning mode which the VM can't poison with an IP of its choosing.
Only the missing of packets is fatal...
For IPv6 the network might use DHCPv6 which we can procss in much the
same
way, or it might be doing stateless autoconfig. So we'd need some host level config parameter to specify whether to learn based on DHCPv6, or based on the router advertisments. In the latter case a VM auto-assigns itself an IPv6 address based on the router prefix + its MAC address. So if we spot the router prefix + know the MAC addr we can safely set the IPv6 addr filter
The issue with looking out for multiple parameters, i.e., IP and IPv6 (as well know address parameters) is that if only one parameter was found a partial instantiation of the filters would be possible. I don't support something like this at the moment, so I would only want to wait for one parameter 'IP'... :-/ Also waiting for IP or IPv6 while the other is already known could take some time... Regards, Stefan
Regards, Daniel -- |: Red Hat, Engineering, London -o-
http://people.redhat.com/berrange/:|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org:| |: http://autobuild.org -o- http://search.cpan.org/~danberr/:| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
participants (3)
-
Daniel P. Berrange
-
Eric Blake
-
Stefan Berger