- Devel - libvirt List Archives

[Libvir] PATCH: support Xen 3.0.5
by Daniel P. Berrange 13 Apr '07

13 Apr '07

I've been doing some testing with current xen-unstable (ie what will very shortly be 3.0.5) and came across a whole bunch of things which needed fixing - some expected, others not expected. The attached patch addresses the following issues: - Many of the hypercalls have their structs changed so that int64_t or 'foo *' members are always 64-bit aligned even on 32-bit platforms. This is part of the work to allow 32-bit Dom0/DomU to work on 64-bit hypervisor. For the int64_t types I had to annotate with __attribute__((aligned(8))). This did not work for pointer data types, so I for those I had to do a more complex hack with union { foo *v; int64_t pad __attribute__((aligned(8))) } This matches what is done in the public (BSD licensed) Xen HV header files. We already had ways to deal with v0 vs v2 hypercalls structs. This change is still techincally v2, but just a minor revision of the domctl or sysctl interfaces. Thus I have named the extra structs v2d5 or v2s3 to indicated hypercall version 2, domctl version 5 or hypercall version 2, sysctl version 3 respectively. - The 'flags' field in the getdomaininfo hypercall now has an extra flag defined '(1<<1)' which was previously not used, is now used to indicate that the guest is HVM. Thus when fetching domain state, we have to mask out that flag, otherwise we'll never match the correct paused/running/ blocked/etc states. - In the xenHypervisorNumOfDomains method, under certain scenarios we will re-try the hypercall, allocating a bigger memory buffer. Well due to the ABI alignment changes we hit that scenario everytime, and ended up allocating a multi-GB buffer :-) The fixed structs sort this out, but as a preventative measure for any future HV changes the patch will break out of the loop at the 10,000 guest mark to avoid allocating GB of memory. - The unified Xen driver broke the GetVCPUs method - it was mistakenly checking for return value == 0, instead of > 0. Trivial fix. - The method to open the XenD connection was calling xenDaemonGetVersion to test if the connection succeeded. But then also calling the xend_detect_config_version which does pretty much same thing. So I removed the former, and now we only do the latter as a 'ping' test when opening. This removes 1 HTTP GET which is worthwhile performance boost given how horrifically slow XenD is. - The HVM SEXPR for configuring the VNC / SDL graphics is no longere part of the (image) block. it now matches the PVFB graphics config and is an explicit (vfb) block within the (devices) block. So if xend_config_format >= 4 we use the new style config - this is assuming upstream XenD is patched to increment xend_config_format from 3 to 4 - I send a patch & am confident it will be applied very shortly. - The QEMU device model allows a user to specify multiple devices for the boot order, eg 'andc' to indicated 'floppy', 'network' 'cdrom', 'disk'. We assumed it was a single letter only. I now serialize this into multiple <boot dev='XXX'/> elements, ordered according to priority. The XML -> SEXPR conversion allows the same. I've tested all this on a 32-bit Dom0 running on 32-bit HV, and 64-bit HV, but not tested a 64-bit Dom0 on 64-bit HV. I'm pretty sure it'll work,but if anyone is runnning 64-on-64 please test this patch. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 13

[Libvir] [PATCH][RFC] shows scheduler information
by Atsushi SAKAI 13 Apr '07

13 Apr '07

Hi, This patch intends add a showing function of scheduler information. If apply this patch to current revision 1.490 and type virsh dominfo 0 then shows following scheduler information. Currently I assume credit scheduler only. getting function is implemented but setting function is not yet. I just waiting for comments. Id: 0 Name: Domain-0 UUID: 00000000-0000-0000-0000-000000000000 OS Type: linux State: running CPU(s): 1 CPU time: 432.7s Max memory: no limit Used memory: 1933312 kB scheduler info 5 256 0 SchedulerID: 5 Weight: 256 Cap: 0 Thanks Atsushi SAKAI

4 26

[Libvir] [PATCH] Check ReadOnly-flag
by S.Sakamoto 13 Apr '07

13 Apr '07

Hi When a general user uses command "# virsh --connect xen start domain", don't become an error, and domain start. Because ReadOnly-flag is not "1" when a general user uses it. So, here's the patch adds to check ReadOnly-flag. Signed-off-by: Shigeki Sakamoto <fj0588di(a)aa.jp.fujitsu.com> Thanks, Shigeki Sakamoto. Index: src/libvirt.c =================================================================== RCS file: /data/cvs/libvirt/src/libvirt.c,v retrieving revision 1.66 diff -u -p -r1.66 libvirt.c --- src/libvirt.c 4 Apr 2007 14:19:49 -0000 1.66 +++ src/libvirt.c 12 Apr 2007 07:55:32 -0000 @@ -301,6 +301,10 @@ do_open (const char *name, int flags) break; } } + + if (flags & VIR_DRV_OPEN_RO) { + ret->flags = VIR_CONNECT_RO; + } return ret;

2 1

[Libvir] [Q] about virsh-connect
by S.Sakamoto 12 Apr '07

12 Apr '07

Hi, I have a question. 　Previously(libvirt 0.2.1) We can do remote connection via URI successfully.. (virsh --connect http://127.0.0.1:8000 list) But Current version(libvirt-current) cannot connect remotely. This event occurs in both general user and administrator. I investigate the code and found folloing code is included (see line 94) If "name" is not "xen", decline connection. ("# virsh --help connect" is output with "<name> hypervisor connection URI") Is this change intentionally? xen_unified.c =========================================================== 84| static int 85| xenUnifiedOpen (virConnectPtr conn, const char *name, int flags) 86| { 87| int i, j; 88| xenUnifiedPrivatePtr priv; 89| 90| /* If name == NULL, name == "", or begins with "xen", then it's for us. / 91| if (!name || name[0] == '\0') 92| name = "xen"; 93| if (strncasecmp (name, "xen", 3) != 0) 94| return VIR_DRV_OPEN_DECLINED; =========================================================== Thanks, Shigeki Sakamoto.

2 1

[Libvir] [PATCH] virsh dumpxml and create fails with no UNAME cdrom device
by Nobuhiro Itou 12 Apr '07

12 Apr '07

Hi, In the xmexample.hvm(/etc/xen/xmexample.hvm), no UNAME cdrom device is written. disk = [ 'file:/var/images/min-el3-i386.img,hda,w', ',hdc:cdrom,r' ] But virsh dumpxml fails if the domain has virtual cdrom device without UNAME. And virsh create fails if the domain config XML has no UNAME cdrom device, too. The attached patch adds to solve this problem. Signed-off-by: Nobuhiro Itou <fj0873gn(a)aa.jp.fujitsu.com> Thanks, Nobuhiro Itou.

2 3

[Libvir] RFC [0/3]: Re-factor QEMU daemon protocol to use XDR
by Daniel P. Berrange 11 Apr '07

11 Apr '07

I've been thinking about a way to move forward on the remote management capabilities in libvirt since our last list discussion didn't come to any clear agreement. The current situation - QEMUD daemon uses a hand-crafted protocol basically just dumping structs on the wire. Key points: - Fairly simple to understand code for marshalling/demarshalling - Very low overhead on wire - Not safe endianess - Not safe wrt to architecture data alignment rules - No authentication/encryption - Server driver specific to QEMU - The generic libvirt daemon uses SunRPC as the underlying protocol. - Protocol versioning - Safe wrt to endianess - Safe wrt to architecture data alignment rules - Added TLS encryption / SSH tunneling - Generic for any libvirt backend - Doesn't work with non-blocking sockets due to SunRPC limitations - SunRPC api is not well known / 'wierd' - Call + reply only; no async signals from server Looking at the current impl of the generic libvirt daemon there are some things which strike me: 1. We have an extra layer of marshalling/demarshalling calls. ie the methods in the libvirt driver, call to the SunRPC marshalling methods, which populate an XDR generated struct, which call the XDR routines to write the data. 2. The main loop doesn't allow us to listen for other non-SunRPC file handles (eg the QEMU/dnsmasq processes). 3. We've basically re-written the the SunRPC svc*_create methods to get support for TLS / SSH We are going to need to address 2 by creating our own mainloop impl. For point 1 we can have the libvirt driver populate the XDR struct directly and eliminate the SunRPC generated marshalling stubs completely. If we did that, and given points 2 & 3, we're basically left using very little of the SunRPC capabilities at all. Pretty much only thing it is doing for us at that point is the protocol versioning, and giving us a horrific API which doesn't support non-blocking IO so forces us to be multi-threads or multi-process. So, I'm thinking why don't we just use XDR directly & be done with it. As an experiment I took the existing QEMU daemon & driver and re-factored the hand written binary protocol to use XDR instead. Looking at the diffstat - there was basically no appreciable difference in codesize after the re-factoring, and we get the added benefit of being endianness & alignment safe. qemud/Makefile.am | 15 + qemud/conf.c | 1 qemud/dispatch.c | 538 ++++++++++++++++------------------------------ qemud/dispatch.h | 2 qemud/internal.h | 21 + qemud/protocol.h | 329 ---------------------------- qemud/protocol.x | 608 ++++++++++++++++++++++++++++++++++++++++++++++++++++ qemud/qemud.c | 189 +++++++++++----- qemud/uuid.c | 1 src/Makefile.am | 11 src/internal.h | 2 src/qemu_internal.c | 593 ++++++++++++++++++++++++++------------------------ 12 files changed, 1288 insertions(+), 1022 deletions(-) The particular changes I'll explain in the next 3 mails where I attach the patches. Meanwhile though, I'm thinking we could take approach of incremental code refactoring, pulling in all the various capabilities from the remote patch (except for the SunRPC demarshalling itself) to the QEMU daemon. At each stage having a working daemon & client but with ever growing capabilities, until it provides full remote management. 1. Add an extra initial message type to do a major/minor version number exchange. Based on the negotiated versions, the server can activate or deactivate particular requests it allows from the client. 2. Add in the URI parsing routines to the qemud_interal.c driver, but only allow use of QEMU initially. 3. Add in the TLS + SSH client & server socket setup code to give an authenticated & secured remote channel 4. Re-factor qemud/dispatch.c so that it calls into the regular libvirt API for the Xen case, but keeps calling qemud/driver.c for the QEMU case. BTW, these patches are not intended to be applied to CVS - they're for discussion purposes at this stage. That said, they are fully functional so you can apply them to your checkout & you should be able run all the existing virsh commands against the libvirt_qemud. Just make sure you patch & rebuild both the client & server ;-) Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 12

[Libvir] [Q] about Missing type converters in python/generator.py
by Atsushi SAKAI 11 Apr '07

11 Apr '07

Hi, I try to make a python bindings. But I run the python/generator.py. The following messages are appeared. (Missing type converters:) Is there any advice to solve this? ================== /usr/lib/python2.4/xmllib.py:9: DeprecationWarning: The xmllib module is obsolete. Use xml.sax instead. warnings.warn("The xmllib module is obsolete. Use xml.sax instead.", DeprecationWarning) Found 79 functions in libvirt-api.xml Found 1 functions in libvirt-python-api.xml Generated 67 wrapper functions, 9 failed, 4 skipped Missing type converters: virErrorPtr:4 virVcpuInfoPtr:1 void *:2 unsigned long *:2 ================== Thanks Atsushi SAKAI

2 2

[Libvir] [PATCH] Add xenUnifiedRegister Comment for xml files
by Atsushi SAKAI 10 Apr '07

10 Apr '07

Hi, I run apibuild.py on docs, following messages are appeared. Missing comment for function xenUnifiedDriver This patch fixes this. If comments are not appropriate, please change it. Thanks Atsushi SAKAI --- libvirt.0410/src/xen_unified.c 2007-04-04 23:19:49.000000000 +0900 +++ libvirt.xenuni/src/xen_unified.c 2007-04-10 13:30:56.000000000 +0900 @@ -762,6 +762,13 @@ static virDriver xenUnifiedDriver = { .domainSetAutostart = xenUnifiedDomainSetAutostart, }; +/** + * xenUnifiedRegister: + * + * Register xen related drivers + * + * Returns the driver priority or -1 in case of error. + */ int xenUnifiedRegister (void) {

2 2

[Libvir] Virtual network iptables rules
by Daniel P. Berrange 06 Apr '07

06 Apr '07

Warning, this is a long & complicated email with lots of horrible details :-) I've long been a little confused with the way iptables & bridging interacts, so set out to do some experiments. I added a -j LOG rule to every single chain in both the filter & nat tables, and then tried various traffic patterns, to see which chains were traversed & in which order. There are 2 types of config I considered - virtual networking, and shared physical device. For both these I tried with net.bridge.bridge-nf-call-iptables on & off. This gave 4 scenarios to test with. For the test I simply did 'ping -c 1 <ip addr>', which gives a simple roundtrip with a single packet in each direction. The results were as follows.... Scenario 1: Virtual network =========================== net.bridge.bridge-nf-call-iptables = 0 Host: eth0 -> Internet virbr0 -> MASQUERADE to eth0 Guest: vif1.0 -> virbr0 Traffic: Guest -> Google ------------------------ Out: NAT-PREROUTING IN=virbr0 OUT= SRC=192.168.122.47 DST=64.233.167.99 FORWARD IN=virbr0 OUT=eth0 SRC=192.168.122.47 DST=64.233.167.99 NAT-POSTROUTING IN= OUT=eth0 SRC=192.168.122.47 DST=64.233.167.99 Back: FORWARD IN=eth0 OUT=virbr0 SRC=64.233.167.99 DST=192.168.122.47 Traffic: Guest -> Host ---------------------- Out: NAT-PREROUTING IN=virbr0 OUT= SRC=192.168.122.47 DST=192.168.122.1 INPUT IN=virbr0 OUT= SRC=192.168.122.47 DST=192.168.122.1 Back: OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 Traffic: Host -> Guest ---------------------- Out: NAT-OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 NAT-POSTROUTING IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 Back: INPUT IN=virbr0 OUT= SRC=192.168.122.47 DST=192.168.122.1 Scenario 2: Virtual network =========================== net.bridge.bridge-nf-call-iptables = 1 Host: eth0 -> Internet virbr0 -> MASQUERADE to eth0 Guest: vif1.0 -> virbr0 Traffic: Guest -> Google ------------------------ Out: NAT-PREROUTING IN=virbr0 OUT= PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99 FORWARD IN=virbr0 OUT=eth0 PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99 NAT-POSTROUTING IN= OUT=eth0 PHYSIN=vif1.0 SRC=192.168.122.47 DST=64.233.167.99 Back: FORWARD IN=eth0 OUT=virbr0 SRC=64.233.167.99 DST=192.168.122.47 Traffic: Guest -> Host ---------------------- Out: NAT-PREROUTING IN=virbr0 OUT= PHYSIN=vif1.0 SRC=192.168.122.47 DST=192.168.122.1 INPUT IN=virbr0 OUT= PHYSIN=vif1.0 SRC=192.168.122.47 DST=192.168.122.1 Back: OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 Traffic: Host -> Guest ---------------------- Out: NAT-OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 OUTPUT IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 NAT-POSTROUTING IN= OUT=virbr0 SRC=192.168.122.1 DST=192.168.122.47 Back: INPUT IN=virbr0 OUT= PHYSIN=vif1.0 SRC=192.168.122.47 DST=192.168.122.1 Scenario 3: Shared physical device ================================== net.bridge.bridge-nf-call-iptables = 0 Host: peth1 -> Internet xenbr0 -> peth1 Guest: vif2.0 -> xenbr0 Traffic: Guest -> Google ------------------------ Nada Traffic: Guest -> Host ---------------------- Out: NAT-PREROUTING IN=eth1 OUT= SRC=192.168.254.120 DST=192.168.254.132 INPUT IN=eth1 OUT= SRC=192.168.254.120 DST=192.168.254.132 Back: OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 Traffic: Host -> Guest ---------------------- Out: NAT-OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 NAT-POSTROUTING IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 Back: INPUT IN=eth1 OUT= SRC=192.168.254.120 DST=192.168.254.132 Scenario 4: Shared physical device ================================== net.bridge.bridge-nf-call-iptables = 1 Host: peth1 -> Internet xenbr0 -> peth1 Guest: vif2.0 -> xenbr0 Traffic: Guest -> Google ------------------------ Out: NAT-PREROUTING IN=xenbr1 OUT= PHYSIN=vif2.0 SRC=192.168.254.120 DST=64.233.167.99 FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=vif2.0 PHYSOUT=peth1 SRC=192.168.254.120 DST=64.233.167.99 NAT-POSTROUTING IN= OUT=xenbr1 PHYSIN=vif2.0 PHYSOUT=peth1 SRC=192.168.254.120 DST=64.233.167.99 Back: FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=peth1 PHYSOUT=vif2.0 SRC=64.233.167.99 DST=192.168.254.120 Traffic: Guest -> Host ---------------------- Out: NAT-PREROUTING IN=xenbr1 OUT= PHYSIN=vif2.0 SRC=192.168.254.120 DST=192.168.254.132 FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=vif2.0 PHYSOUT=vif0.1 SRC=192.168.254.120 DST=192.168.254.132 NAT-POSTROUTING IN= OUT=xenbr1 PHYSIN=vif2.0 PHYSOUT=vif0.1 SRC=192.168.254.120 DST=192.168.254.132 INPUT IN=eth1 OUT= SRC=192.168.254.120 DST=192.168.254.132 Back: OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=vif0.1 PHYSOUT=vif2.0 SRC=192.168.254.132 DST=192.168.254.120 Traffic: Host -> Guest ---------------------- Out: NAT-OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 OUTPUT IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 NAT-POSTROUTING IN= OUT=eth1 SRC=192.168.254.132 DST=192.168.254.120 FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=vif0.1 PHYSOUT=vif2.0 SRC=192.168.254.132 DST=192.168.254.120 Back: FORWARD IN=xenbr1 OUT=xenbr1 PHYSIN=vif2.0 PHYSOUT=vif0.1 SRC=192.168.254.120 DST=192.168.254.132 INPUT IN=eth1 OUT= SRC=192.168.254.120 DST=192.168.254.132 Now in this email I'm really only concerned with the first 2 virtual network scernaios. The shared physical device stuff can be ignored henceforth, basically because it 'just works(tm)'. For virtual networks there are basically 3 types of networking config we need to represent in terms of iptables rules, and these need to work for scenrios 1 & 2 - ie regardless of the magic sysctl knob. Here is what we currently implement...... Type 1: Isolated virtual network -------------------------------- - We don't add anything here Type 2: Forwarding to a specific NIC only ----------------------------------------- Chain POSTROUTING (policy ACCEPT 345 packets, 32627 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * eth1 0.0.0.0/0 0.0.0.0/0 PHYSDEV match ! --physdev-is-bridged Chain FORWARD (policy ACCEPT 29 packets, 2244 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- eth1 vnet0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 0 0 ACCEPT all -- vnet0 eth1 0.0.0.0/0 0.0.0.0/0 0 0 ACCEPT all -- * eth1 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in vnet0 Chain INPUT (policy ACCEPT 80483 packets, 382M bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- vnet0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- vnet0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 0 0 ACCEPT udp -- vnet0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- vnet0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 Type 3: Forwarding to any active NIC ------------------------------------ Chain POSTROUTING (policy ACCEPT 360 packets, 33843 bytes) pkts bytes target prot opt in out source destination 2 476 MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match ! --physdev-is-bridged Chain FORWARD (policy ACCEPT 29 packets, 2244 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 0 0 ACCEPT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 PHYSDEV match --physdev-in virbr0 Chain INPUT (policy ACCEPT 80884 packets, 382M bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 So how do these shape up, given the traversal scenarios & the overall desire to be as restrictive as possible with traffic. Problem: The INPUT rules are missing altogether for the isolated virtual network so potentially DHCP/DNS will be blocked Solution: Add them - simple bug. Problem: The POSTROUTING rule is too generic so it matches pretty much any kind of traffic, from any virtual network, or even from VPN devices setup by VPNC. Solution: Only masquerade traffic whose source address is within the netmask associated with the virtual network in question Problem: The FORWARD rule is too generic, forwarding traffic to/from the virtual network regardless of whether the dest/src IP address is within the netmask associated with the virtual network. Assuming the first problem is setup to only masquerade valid IP addresses from the virtual network, this rule would then allow guests to spoof their IP and have it forwarded off-host. Solution: Only forward packets whose IP address is within the netmask associated with the virtual network Problem: The policy of the FORWARD rule is ACCEPT, and/or later user defined rules may inadvertently match on traffic from the virtual network, again allowing through spoof traffic, or traffic from what should be an isolated virtual network Solution: There needs to be a catch-all REJECT rule associated with every bridge device, in both directions Problem: There is an extra physdev match per bridge device, and per guest device. This is basically unneccessary since the previous rule sets will already have allowed through the traffic. The physdev matches also only work if net.bridge.bridge-nf-call-iptables = 1 Solution: Simply remove the per-device matches Problem: The POSTROUTING rule has a physdev match applied, which only works if net.bridge.bridge-nf-call-iptables = 1. Solution: Remove physdev match & masquerade based on network address associated with the virtual network If we apply all solution outlined here, we'll end up with a set of rules which look like this.......... Type 1: Isolated virtual network -------------------------------- Chain POSTROUTING (policy ACCEPT 273 packets, 26341 bytes) pkts bytes target prot opt in out source destination Chain FORWARD (policy ACCEPT 29 packets, 2244 bytes) pkts bytes target prot opt in out source destination 0 0 REJECT all -- * vnet2 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 0 0 REJECT all -- vnet2 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain INPUT (policy ACCEPT 76724 packets, 366M bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- vnet2 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- vnet2 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 0 0 ACCEPT udp -- vnet2 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- vnet2 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 Type 2: Forwarding to a specific NIC only ----------------------------------------- Chain POSTROUTING (policy ACCEPT 273 packets, 26341 bytes) pkts bytes target prot opt in out source destination 0 0 MASQUERADE all -- * eth1 192.168.200.0/24 0.0.0.0/0 Chain FORWARD (policy ACCEPT 29 packets, 2244 bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- eth1 vnet3 0.0.0.0/0 192.168.200.0/24 state RELATED,ESTABLISHED 0 0 ACCEPT all -- vnet3 eth1 192.168.200.0/24 0.0.0.0/0 0 0 REJECT all -- * vnet3 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 0 0 REJECT all -- vnet3 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain INPUT (policy ACCEPT 76724 packets, 366M bytes) pkts bytes target prot opt in out source destination 0 0 ACCEPT udp -- vnet3 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- vnet3 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 0 0 ACCEPT udp -- vnet3 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- vnet3 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 Type 3: Forwarding to any active NIC ------------------------------------ Chain POSTROUTING (policy ACCEPT 273 packets, 26341 bytes) pkts bytes target prot opt in out source destination 16 1292 MASQUERADE all -- * * 192.168.122.0/24 0.0.0.0/0 Chain FORWARD (policy ACCEPT 29 packets, 2244 bytes) pkts bytes target prot opt in out source destination 44 20200 ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 state RELATED,ESTABLISHED 56 3676 ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0 0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable 0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Chain INPUT (policy ACCEPT 76724 packets, 366M bytes) pkts bytes target prot opt in out source destination 28 1728 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 5 1640 ACCEPT udp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:67 So in summary: - Every single network type has the 4 INPUT rules for DHCP/DNS - Every single network type has catch all REJECT rules in FORWARD chain for both directions of traffic - A network forwarding to any device, has ACCEPT rules which allow through traffic associated with the virtual network IP range to/from any device - A network forwarding to a specific device, has ACCEPT rules which allow through traffic associated with the virtual network IP range to/from that specific device. - A network forwarding to any device, has MASQUERADE rule to translate source address which matches the virtual network & destined for any dev - A network forwarding to a specific device, has MASQUERADE rule to translate source address which matches the virutal network & destinaed for that specific device - There are no physdev matches needed. Hopefully at least one person has read this far through the email and still understands what is going on.... I'm attaching a patch which implements all this. BTW, there is also a bug in the vif-bridge script for Xen which adds a per-guest VIF physdev match rule. This needs to be removed too. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 12

[Libvir] [PATCH] Remote patch, 2007-04-04
by Richard W.M. Jones 05 Apr '07

05 Apr '07

This is a version of the remote patch -- I'm going to call it "oldremote" because it includes the SunRPC code which we agreed to drop in favour of just using XDR (see [1]). So this patch is just posted here for public record. It applies against the latest CVS. Rich. [1] https://www.redhat.com/archives/libvir-list/2007-March/thread.html#00333 -- Emerging Technologies, Red Hat http://et.redhat.com/~rjones/ 64 Baker Street, London, W1U 7DF Mobile: +44 7866 314 421 Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (USA), Charlie Peters (USA) and David Owens (Ireland)

1 0