May 2013 - Devel - libvirt List Archives

[libvirt] [PATCH 0/2] avoid getpwuid_r deadlock
by Eric Blake 04 Jul '13

04 Jul '13

https://bugzilla.redhat.com/show_bug.cgi?id=964358 Posting now to get reviews started. I'd especially like to get feedback that it doesn't break LXC, and that it works with root-squash NFS when using qemu:qemu instead of root:root in /etc/libvirt/qemu.conf. I hope to do more testing myself, and also see if I can try writing an LD_PRELOAD shim as part of 'make check' to make it easier to test that the right system calls are made during the sequence. Eric Blake (2): util: add virGetGroupList util: make virSetUIDGID async-signal-safe configure.ac | 7 +-- src/libvirt_private.syms | 1 + src/security/security_dac.c | 16 ++++-- src/util/vircommand.c | 10 +++- src/util/virfile.c | 30 +++++++++-- src/util/virutil.c | 123 ++++++++++++++++++++++++++++++-------------- src/util/virutil.h | 7 ++- 7 files changed, 142 insertions(+), 52 deletions(-) -- 1.8.1.4

2 6

[libvirt] [PATCH v3 00/12] Add user namespace support for libvirt lxc
by Gao feng 03 Jul '13

03 Jul '13

This patchset try to add userns support for libvirt lxc. Since userns is nearly completed in linux-3.9, the old kernel doesn't support userns, I add some New XML elements to let people decide if enable userns.The userns is enabled only when user configure the XML. The format of user namespace related XML file like below: <idmap> <uid start='0' target='1000' count='10'> <gid start='0' target='1000' count='10'> </idmap> it means the user in container (which uid:gid is 0:0) will be mapped to the user in host (uid:gid is 1000:1000), count is used to form an u/gid range: The users in container which uid in [start, start + count -1] will be mapped. You can have multiple lines to map differnet id ranges, caution, you must make sure the root user of container has been mapped. This patchset also does the below jobs. 1, Because the uninit userns has no right to create devices, we should create devices for container on host. 2, Changes the owner of fuse and tty device. Change from v2: 1, Mount tmpfs on /stateDir/domain.dev 2, Create devices under /stateDir/doamin.dev/ 3, Mount Move the /.oldroot/stateDir/doamin.dev/ on the /dev/ of container 4, Enhance the configuration, disallow the semi configuration Gao feng (12): LXC: Introduce New XML element for user namespace LXC: enable user namespace only when user set the uidmap LXC: sort the uidmap/gidmap of domain LXC: introduce virLXCControllerSetupUserns and lxcContainerSetID LXC: Creating devices for container on host side LXC: Move creating /dev/ptmx to virLXCControllerSetupDevPTS LXC: fuse: Change files owner to the root user of container LXC: controller: change the owner of tty devices to the root user of container LXC: controller: change the owner of /dev to the root user of container LXC: controller: change the owner of devices created on host LXC: controller: change the owner of /dev/pts and ptmx to the root of container LXC: introduce virLXCControllerChown docs/formatdomain.html.in | 23 ++++ docs/schemas/domaincommon.rng | 31 +++++ src/conf/domain_conf.c | 115 ++++++++++++++++++ src/conf/domain_conf.h | 22 ++++ src/lxc/lxc_container.c | 183 ++++++++++++++-------------- src/lxc/lxc_controller.c | 271 +++++++++++++++++++++++++++++++++++++++++- src/lxc/lxc_fuse.c | 6 + 7 files changed, 557 insertions(+), 94 deletions(-) -- 1.8.1.4

6 49

[libvirt] RFC: Improving unit test coverage
by Daniel P. Berrange 27 Jun '13

27 Jun '13

Over the many years that libvirt has been in existance, we have developed a huge number of features, and rate of change in the codebase shows no sign of slowing down. Unfortunately the rate of regressions and/or new bugs also shows no sign of decreasing. As libvirt is used ever more widely, poor quality releases are having an increasingly serious negative impact on the impressions of libvirt by downstream consumers. In addition the introduction of the new access control facilities will have the unfortunate side-effect of increasing the severity of many bugs. Historically a read-write connection to libvirtd could be considered to be equivalent to root, so a bug affecting an authenticated user would not be considered a privilege escalation, merely denial-of-service. With the introduction of access control, a read-write connection to libvirtd may have substantially fewer privileges than root. The result is that many (arguably all) crasher bugs in libvirtd which were previously considered mere denial of service issues, and not worthy of security alerts, will now be classed as potential privilege escalation flaws. It is clear that the current rate of defects in libvirt code is no longer tenable / acceptable. There are a number of test suites which are targetting libvirt, but this document will focus on the unit testing framework, rather than the integration test harnesses. The rationale for this is - Unit tests are run by all developers daily via 'make check' - Unit tests gate the release process - Unit tests are run every time any RPM is built - Unit tests do not have an dependancy on external system infrastructure / setup - Unit tests are directly synchronized with the code commits since they're in the same GIT repo Unit testing techniques ======================= One of the challenges of unit testing is how to isolate the test suite from the broader environment of the machine being used for development. For example, the test suite must not rely on information located in the filesystem outside the source treee (eg it must not touch /proc or /sys). In addition it must be careful not to negatively impact anything running on the developers' machine (eg it must not spawn libvirtd in a way which would clash with existing running libvirtd) There is no silver bullet to achieve isolation of the test suite from the host OS environment. Instead a variety of different techniques must be applied depending on the scenario to be addressed. Code structure -------------- First, and foremost, code structure has an enourmous impact on the ability to unit test code. Ensuring there are clear boundaries & interfaces between different areas of cdoe is key to allowing pieces to be unit tested in isolation from each other. For example, testing of the QEMU command line generator, can only be achieved because this code for converting from XML to ARGV has been isolated from the code which actually starts up the QEMU process. While some areas of libvirt code are very well modularized, others are not. When writing new unit tests it is not at all unexpected to have to re-factor existing code to further modularize it. Such refactoring is usually good for overall readability of the code, as well as facilitating testing. Intermediate data forms ----------------------- As hinted at above, a key idea when refactoring code to make it more amenable to testing is to introduce an intermediate data form. For example, when generating the QEMU command line arguments from an XML document, the virCommandPtr object serves as the intermediate data form. Instead of having the API which builds the command line, directly execute the QEMU binary, it spits out a virCommandPtr object as an intermediate data form. The contents of this object can then be validated against pre-defined expected command lines. As a counter-example, the network filter code is very hard to unit test as it is currently designed because it does not have any intermediate data form. The methods which process the network filter configuration build up command lines and then directly execute them as they go. There is no way to extract the command line data, without them being executed. To enable unit testing it is neccessary to refactor the network filter code to introduce a clear intermediate data form which can represent the commands that will be executed. Static analysis --------------- While the elephant in the room is Coverity, there is a large amount of static analysis that can be done directly in the libvirt source tree. Indeed all of the 'syntax-check' rules are considered to be static analysis. While these rules all focus on style issues, there is no reason why this should be the case. What makes static analysis tools like Coverity so hard to write is the problem of understanding the semantics of C code. Fortunately, when doing static analysis of a specific project (like libvirt), there is often no need to solve the general case. If there are certain style patterns used in a project, these can make it possible to write project specific analysis rules with little more than a few regular expressions or other simple scripting statements. For example, the two style checks 'src/check-drivername.pl' 'src/check-driverimpls.pl' enforce a consistent naming convention on each internal driver method. With this naming pattern enforced, it is now possible to write a script which validates that every driver method includes a call to the correct access control hook. This is project-specific static analysis that could not be done with Coverity. For more advanced problems it is possible to make use of a framework such as CIL to perform static analysis. CIL is an OCaml project which can parse most C code providing an in-memory representation of its structure. Then it has a number of modules to perform data flow analysis on the code. Combined with knowledge of project coding patterns this tool allows the creation of scripts to validate mutex lock acquisition ordering rules, or correct error reporting on all error exit paths. Monkey patching / Duck Punching ------------------------------- One of the pain-points with a project such as libvirt is finding a way to test parts of the code which rely on external system state, such as /proc, or /sys filesystem data. It is not always possible to refactor the code to test it in isolation from the system services. In such cases, the technique of monkey patching or duck punching (better known from the world of scripting languages) can in fact be used from C. The idea behind the technique is to replace the APIs / objects that are used by the test suite, with set of "mock" objects or API implementations which simulate their operation, without relying on external state. The trick to applying this technique in the C world with native code, is to rely on the use of the dynamic linkers' library preload facility, via the LD_PRELOAD. Any symbol exported by a library listed in the LD_PRELOAD env variable, will override symbols with the same name in the library that the test suite is actually linked to. So, for example, if the preloaded library exports an 'open' symbol, this will replace the 'open' symbol exported by the standard C library. To use this technique it is neccessary to first understand what interfaces need to be replaced. If attempting to test some code which uses files in some location under /sys/, one might change the implementation of 'open', so that when passed a filename starting with '/sys', it will re-write the path to point to a location under the test suite directory. This technique is used in the vircgrouptest.c / vircgroupmock.c files to allow comprehensive testing of libvirt's cgroup management code, without actually ever touching the cgroup filesystem. It is also used by the securityselinuxtest.c / securityselinuxhelper.c files to stub out the sys calls which write SELinux attributes. Fuzz testing ------------ This refers to the technique of taking an API or interface and giving it some data which has some intentionale inaccuracies or edge cases. For example, a API accepting a 'const char *str' might be given a NULL string, or a zero length string, or a string containing non-printable control characters. An API accepting an 'int' might be tested with various boundary conditions such as -1, 0, -MAX_INT, +MAX_INT. In the case of an XML document, one technique would be to incrementally remove attributes and/or elements to see if the parser will ever crash on a NULL pointer (it should either succeed or return a error, never crash). Or alternatively change the values in various attributes as one would with parameters passed to an API. If an attribute appears to be a number, then replace it with a string. If an attribute appears to be an enum, replace it with some bogus value. Try zero length strings for various attributes, or again try non-printable control characters. In the case of an RPC protocol, the idea would be to make (intelligent) changes to packets in an attempt to trick the server into accepting bad data, or even crashing. It is not practical to test all possible wierd data sets, so the trick with fuzz testing is to try to intelligently tailor the data to the interface being tested. eg if a string parameter represents a file path, there's no point just randomly changing characters in the path, since that'll just generate loads of other valid file paths. Instead it is neccessary to make some more planned changes like removing the leading '/' to produce a relative path, inserting a number of '../../../' dirs to trick the application into writing to the incorrect location, and things of that nature. For any moderately complex parser, intelligently designed fuzz testing is likely to identify a wide range of bugs. Error testing ------------- Validating correct operation is great, but there is plenty of code which lives in error handling code paths, which will never be exercised this way. Since they are seldom run, and by their nature deal with unexpected data or events, the error paths are often a good source of bugs. The fuzz testing technique is one very effective way of exercising error handling paths, but should not be considered the only possible one. The monkey patching techniques can also be applied to the task of error checking. Alternatively the code structure may facilitate the injection of error conditions in a well defined manner. For example, all libvirt memory allocation goes via a wrapper API, instead of to malloc() directly. It is possible to tell the wrapper API to fail specific allocation attempts. By running the test suite repeatedly, failing a different memory allocation each time, it is possible to exhaustively test error handling of the tested code. Areas requiring testing ======================= The best way to identify areas lacking code coverage is to simply pick a random libvirt source code file. The odds are that it has insufficient testing. The more optimistic way is to look at the code coverage reports. Run 'autogen.sh' with the --enable-test-coverage arg. Then build it and run 'make check'. Once the tests have completed, then run 'make cov' to generate an HTML code coverage report. Some known areas needing work are: Cgroups setup for QEMU ---------------------- For a given QEMU guest XML configuration there is a well defined set of cgroups parameters that are expected to be set. The vircgroupmock.c file demonstrates how to monkey patch the cgroups filesystem, allowing unit tests to be written. The vircgroupmock.c helper could be reused to create a test suite for validating the operation of the qemu_cgroups.c APIs. Cgroups setup for LXC ---------------------- The same as the QEMU test, but for the lxc_cgroups.c file. SELinux security driver labelling --------------------------------- For a given XML configuration, there is a well defined set of file labels that the SELinux security driver must apply to the filesystem on VM startup. The securityselinuxhelper.c file demonstrates how to monkey patch the SELinux library file labelling APIs from unit tests. Currently the securityselinuxlabeltest.c test suite does checks for labels set on disks and serial devices. There are many other aspects of domain XML that imply labelling. In particular PCI/USB host devices & kernel/initrd boot images. It is also neccessary to deal with backing files for qcow2 format disks. Audit messages -------------- Upon QEMU or LXC guest startup a number of audit records must be logged detailing various pieces of metadata about the guest. By monkey patching the audit_log_user_message() and audit_open() library APIs with an LD_PRELOAD library, it would be possible to perform unit testsing of the domain_audit.c APIs to ensure that correct audit records are logged for various different guest configurations. Network filter -------------- As described earlier, the network filter code is really badly designed from the point of view of unit testing. It needs to be refactored to have an intermediate data form that represents the list of iptables/ebtables rules that are expected for a given XML configuration. Currently it generates shell scripts containing iptables commands, optionally re-directing via the 'firewall-cmd' program for integration with firewalld. Refactoring to unit test this offers a prime opportunity to remove the use of shell entirely The core observation of the nwfilter code is that it consists of lists of commands to be run to create ebtables/iptables rules. Periodically there are commands which are able to rollback the state changes made by earlier commands when an error occurs. The idea for the intermediate data form would thus be to introduce the concept of a "script" object, as a higher level interface than that offered by virCommandPtr. A script would comprise one or more sets of command lines for making changes. At various points it would be possible to add a checkpoint marker, together with one or more sets of command lines that would be execute when errors occur which require rollback. The network filter code would thus output "script" objects whose command sets could be validated by a unit test suite. These script objects can then be translated into virCommandPtr objects for actual execution by separate code. Or even translated into DBus API calls for direct communication with firewalld, bypassing the inefficient firewall-cmd program. The unit test would consist of XML data files with network filter configuration rules, alongside expected script command lines, similar to how the qemu xml -> argv test works. virsh testing ------------- The virsh command has rarely had any serious direct testing of its own. Its code is mostly exercised indirectly by test suites doing testing of QEMU or other drivers. Libvirt has a hypervisor driver that is explicit targetted to testing tools like virsh - the "test" driver. This is a 100% in-memory driver that stubs out all the libvirt APIs with semi-serious semantics. It is used to do automated testing of large parts of virt-install and virt-manager. It can be used to get near 100% testing coverage of the virsh command from within unit tests. Since it is all in-memory there is no scope for it to be affected the host system state. libvirtd testing ---------------- Testing libvirtd is a tricky proposition, because merely starting the daemon will have an effect on host OS state, particularly if any resources are set to autostart. It can also clash with any libvirtd process the developer is already running. The test driver is able to be run inside libvirtd. If it were possible to disable all the other hypervisor drivers in libvirtd, the test driver could be used to execise the RPC dispatch code in libvirtd in a safe contained manner. This could be particularly good for doing scalability testing of libvirtd when large numbers of objects. eg since the test driver doesn't actually run anything, it is possible to start many 1000's of "test" guests with the only memory overhead being that of the parsed XML configuration. This would allow the test suite to identify & validate limits in the RPC protocol or inefficiencies in libvirtd. lock driver client ------------------ The virtlockd client code recently had a bug where it generated invalid RPC packets in response to certain operations. It is not desirable to have the unit tests actually run the full virtlockd daemon, just to validate the client code. It would, however, be possible to have a unit test which provided a fake virtlockd UNIX domain socket for the client to connect to. It would blindly read any data off the socket & validate it against pre-defined RPC packets. This approach is already used with some success in the QEMU JSON monitor test suite. QEMU disk snapshots ------------------- The QEMU snapshot code is some of the most complex code in the QEMU driver. As such it would be desirable to get test coverage of it. The easiest way todo this would be via the libvirt TCK integration test harness. There is, however, value in refactoring some of the code to allow its testing in a unit test environment. One approach would be for the test suite to directly invoke methods in the qemu driver dispatch table. This would require refactoring the qemuStateInitialize code though, to make it possible to setup the basic QEMU driver global state. More desirable would be to refactor the snapshot code so that it was more detached from the public driver APIs. For example if some of it could be isolated in a qemu_snapshot.c file, this may make it practical to unit test some of it. Techniques from the qemumonitorjsontest.c file could be used to stub out the monitor APIs that the snapshot code requires in order to operate. QEMU driver startup ------------------- An important aspect of the QEMU driver is that it is able to reconnect to existing domains when it starts up. It would be desirable to unit test this functionality to validate the sequence of operations done when reconnecting to a guest. Again this would require some refactoring of the qemuStateInitialize method to allow its execution in a test suite. It would also likely require the techniques for stubing the QEMU monitor APIs. XML parser/formatter reliability -------------------------------- While we have good testing of valid XML documents, there have been a number of cases where invalid XML documents have caused crashes or other bad behaviour. It is not reasonable to add test cases for every possible invalid XML document. Instead we need to adopt a fuzzing approach, where we take valid XML documents and then make incremental changes to them & then ensure that the code does not crash. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

2 1

[libvirt] [PATCHv4] Configure native vlan modes on Open vSwitch ports
by james robson 26 Jun '13

26 Jun '13

This patch adds functionality to allow libvirt to configure the 'native-tagged' and 'native-untagged' modes on openvswitch networks. v2 changes: Fix problems reported by Eric Blake v3 changes: Re work patch to address review comments v4 changes: Use enum for native modes --- diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index 9284534..ba32185 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3498,6 +3498,13 @@ qemu-kvm -net nic,model=? /dev/null <parameters interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> </virtualport> </interface> + <interface type='bridge'> + <vlan trunk='yes'> + <tag id='42'/> + <tag id='123' nativeMode='untagged'/> + </vlan> + ... + </interface> <devices> ...</pre> @@ -3524,6 +3531,15 @@ qemu-kvm -net nic,model=? /dev/null vlan element. + + For network connections using openvswitch it is possible to + configure the 'native-tagged' and 'native-untagged' vlan modes + (Since 1.0.5). This uses the optional + <code>nativeMode</code> attribute on the <code><tag></code> + element: <code>nativeMode</code> may be set to 'tagged' or + 'untagged'. The id atribute of the element sets the native vlan. + + <h5><a name="elementLink">Modifying virtual link state</a></h5> <pre> ... diff --git a/docs/formatnetwork.html.in b/docs/formatnetwork.html.in index a1198ce..29e12d9 100644 --- a/docs/formatnetwork.html.in +++ b/docs/formatnetwork.html.in @@ -446,6 +446,13 @@ <parameters interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/> </virtualport> </interface> + <interface type='bridge'> + <vlan trunk='yes'> + <tag id='42'/> + <tag id='123' nativeMode='untagged'/> + </vlan> + ... + </interface> <devices> ...</pre> @@ -469,6 +476,14 @@ be added to the vlan element. + For network connections using openvswitch it is possible to + configure the 'native-tagged' and 'native-untagged' vlan modes + (Since 1.0.6). This uses the optional + <code>nativeMode</code> attribute on the <code><tag></code> + element: <code>nativeMode</code> may be set to 'tagged' or + 'untagged'. The id atribute of the element sets the native vlan. + + <code><vlan></code> elements can also be specified in a <code><portgroup></code> element, as well as directly in a domain's <code><interface></code> element. In the case diff --git a/docs/schemas/networkcommon.rng b/docs/schemas/networkcommon.rng index 51ff759..e60f1fc 100644 --- a/docs/schemas/networkcommon.rng +++ b/docs/schemas/networkcommon.rng @@ -204,6 +204,14 @@ <param name="maxInclusive">4095</param> </data> </attribute> + <optional> + <attribute name="nativeMode"> + <choice> + <value>tagged</value> + <value>untagged</value> + </choice> + </attribute> + </optional> <empty/> </element> </oneOrMore> diff --git a/src/conf/netdev_vlan_conf.c b/src/conf/netdev_vlan_conf.c index 13ba8c6..2b4cd48 100644 --- a/src/conf/netdev_vlan_conf.c +++ b/src/conf/netdev_vlan_conf.c @@ -17,6 +17,7 @@ * * Authors: * Laine Stump <laine(a)redhat.com> + * James Robson <jrobson(a)websense.com> */ #include <config.h> @@ -27,12 +28,15 @@ #define VIR_FROM_THIS VIR_FROM_NONE +VIR_ENUM_IMPL(virNativeVlanMode, VIR_NATIVE_VLAN_MODE_LAST, "default", "tagged", "untagged") + int virNetDevVlanParse(xmlNodePtr node, xmlXPathContextPtr ctxt, virNetDevVlanPtr def) { int ret = -1; xmlNodePtr save = ctxt->node; const char *trunk = NULL; + const char *nativeMode = NULL; xmlNodePtr *tagNodes = NULL; int nTags, ii; @@ -54,6 +58,8 @@ virNetDevVlanParse(xmlNodePtr node, xmlXPathContextPtr ctxt, virNetDevVlanPtr de goto error; } + def->nativeMode = 0; + def->nativeTag = 0; for (ii = 0; ii < nTags; ii++) { unsigned long id; @@ -68,6 +74,19 @@ virNetDevVlanParse(xmlNodePtr node, xmlXPathContextPtr ctxt, virNetDevVlanPtr de _("vlan tag id %lu too large (maximum 4095)"), id); goto error; } + if ((nativeMode = virXPathString("string(./@nativeMode)", ctxt)) != NULL) { + if (def->nativeMode != 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("duplicate native vlan setting")); + goto error; + } + if ((def->nativeMode = virNativeVlanModeTypeFromString(nativeMode)) <= 0) { + virReportError(VIR_ERR_XML_ERROR, + _("Attribute \"nativeMode='%s'\" is invalid"), nativeMode); + goto error; + } + def->nativeTag = id; + } def->tag[ii] = id; } @@ -89,6 +108,12 @@ virNetDevVlanParse(xmlNodePtr node, xmlXPathContextPtr ctxt, virNetDevVlanPtr de "is required for more than one vlan tag"), trunk); goto error; } + if (def->nativeMode != 0) { + virReportError(VIR_ERR_XML_ERROR, "%s", + _("invalid configuration in <vlan> - \"trunk='no'\" is " + "not allowed with a native vlan id")); + goto error; + } /* allow (but discard) "trunk='no' if there is a single tag */ if (STRCASENEQ(trunk, "no")) { virReportError(VIR_ERR_XML_ERROR, @@ -125,7 +150,17 @@ virNetDevVlanFormat(virNetDevVlanPtr def, virBufferPtr buf) virBufferAsprintf(buf, "<vlan%s>\n", def->trunk ? " trunk='yes'" : ""); for (ii = 0; ii < def->nTags; ii++) { - virBufferAsprintf(buf, " <tag id='%u'/>\n", def->tag[ii]); + if (def->nativeMode != VIR_NATIVE_VLAN_MODE_DEFAULT && def->nativeTag == def->tag[ii]) { + /* check the nativeMode in case we get <tag id='0'/>*/ + const char *mode = virNativeVlanModeTypeToString(def->nativeMode); + if (!mode) { + virReportError(VIR_ERR_INTERNAL_ERROR, "%s", + _("Bad vlaue for nativeMode")); + } + virBufferAsprintf(buf, " <tag id='%u' nativeMode='%s'/>\n", def->tag[ii], mode); + } else { + virBufferAsprintf(buf, " <tag id='%u'/>\n", def->tag[ii]); + } } virBufferAddLit(buf, "</vlan>\n"); return 0; diff --git a/src/util/virnetdevopenvswitch.c b/src/util/virnetdevopenvswitch.c index 2aee445..47e6027 100644 --- a/src/util/virnetdevopenvswitch.c +++ b/src/util/virnetdevopenvswitch.c @@ -109,8 +109,22 @@ int virNetDevOpenvswitchAddPort(const char *brname, const char *ifname, virCommandAddArgList(cmd, "--timeout=5", "--", "--may-exist", "add-port", brname, ifname, NULL); - if (virBufferUse(&buf) != 0) + if (virBufferUse(&buf) != 0) { + switch (virtVlan->nativeMode) { + case VIR_NATIVE_VLAN_MODE_TAGGED: + virCommandAddArg(cmd, "vlan_mode=native-tagged"); + virCommandAddArgFormat(cmd, "tag=%d", virtVlan->nativeTag); + break; + case VIR_NATIVE_VLAN_MODE_UNTAGGED: + virCommandAddArg(cmd, "vlan_mode=native-untagged"); + virCommandAddArgFormat(cmd, "tag=%d", virtVlan->nativeTag); + break; + case VIR_NATIVE_VLAN_MODE_DEFAULT: + default: + break; + } virCommandAddArgList(cmd, virBufferCurrentContent(&buf), NULL); + } if (ovsport->profileID[0] == '\0') { virCommandAddArgList(cmd, diff --git a/src/util/virnetdevvlan.c b/src/util/virnetdevvlan.c index 2fe2017..eed32f7 100644 --- a/src/util/virnetdevvlan.c +++ b/src/util/virnetdevvlan.c @@ -33,6 +33,8 @@ virNetDevVlanClear(virNetDevVlanPtr vlan) { VIR_FREE(vlan->tag); vlan->nTags = 0; + vlan->nativeMode = 0; + vlan->nativeTag = 0; } void @@ -54,7 +56,9 @@ virNetDevVlanEqual(const virNetDevVlanPtr a, const virNetDevVlanPtr b) return false; if (a->trunk != b->trunk || - a->nTags != b->nTags) { + a->nTags != b->nTags || + a->nativeMode != b->nativeMode || + a->nativeTag != b->nativeTag) { return false; } @@ -89,6 +93,8 @@ virNetDevVlanCopy(virNetDevVlanPtr dst, const virNetDevVlanPtr src) dst->trunk = src->trunk; dst->nTags = src->nTags; + dst->nativeMode = src->nativeMode; + dst->nativeTag = src->nativeTag; memcpy(dst->tag, src->tag, src->nTags * sizeof(*src->tag)); return 0; } diff --git a/src/util/virnetdevvlan.h b/src/util/virnetdevvlan.h index c6b16ef..a084938 100644 --- a/src/util/virnetdevvlan.h +++ b/src/util/virnetdevvlan.h @@ -18,16 +18,28 @@ * Authors: * Laine Stump <laine(a)redhat.com> */ - +# include <virutil.h> #ifndef __VIR_NETDEV_VLAN_H__ # define __VIR_NETDEV_VLAN_H__ +typedef enum { + VIR_NATIVE_VLAN_MODE_DEFAULT = 0, + VIR_NATIVE_VLAN_MODE_TAGGED, + VIR_NATIVE_VLAN_MODE_UNTAGGED, + + VIR_NATIVE_VLAN_MODE_LAST +} virNativeVlanMode; + +VIR_ENUM_DECL(virNativeVlanMode) + typedef struct _virNetDevVlan virNetDevVlan; typedef virNetDevVlan *virNetDevVlanPtr; struct _virNetDevVlan { bool trunk; /* true if this is a trunk */ int nTags; /* number of tags in array */ unsigned int *tag; /* pointer to array of tags */ + int nativeMode; + unsigned int nativeTag; }; void virNetDevVlanClear(virNetDevVlanPtr vlan); diff --git a/tests/networkxml2xmlin/openvswitch-net.xml b/tests/networkxml2xmlin/openvswitch-net.xml index a3d82b1..2f6084d 100644 --- a/tests/networkxml2xmlin/openvswitch-net.xml +++ b/tests/networkxml2xmlin/openvswitch-net.xml @@ -21,4 +21,13 @@ <parameters profileid='alice-profile'/> </virtualport> </portgroup> + <portgroup name='native'> + <vlan trunk='yes'> + <tag id='123' nativeMode='tagged'/> + <tag id='444'/> + </vlan> + <virtualport> + <parameters profileid='native-profile'/> + </virtualport> + </portgroup> </network> diff --git a/tests/networkxml2xmlout/openvswitch-net.xml b/tests/networkxml2xmlout/openvswitch-net.xml index a3d82b1..2f6084d 100644 --- a/tests/networkxml2xmlout/openvswitch-net.xml +++ b/tests/networkxml2xmlout/openvswitch-net.xml @@ -21,4 +21,13 @@ <parameters profileid='alice-profile'/> </virtualport> </portgroup> + <portgroup name='native'> + <vlan trunk='yes'> + <tag id='123' nativeMode='tagged'/> + <tag id='444'/> + </vlan> + <virtualport> + <parameters profileid='native-profile'/> + </virtualport> + </portgroup> </network> Protected by Websense Hosted Email Security -- www.websense.com

3 7

[libvirt] [PATCH 00/11] Support CHAP authentication for iscsi pool
by Osier Yang 20 Jun '13

20 Jun '13

The XMLs like (<auth type='chap' login='foo' passwd='kudo'/>) was introduced long ago, but it's never used for any pool backend. This implements the support first (See 6/11 for details), and based on it, using "secret" object for the authentication is added too. E.g. <auth type='chap' username='foo'> <secret uuid='48dcd4a4-b25f-4fc6-8874-84797c6e3678'/> </auth> Osier Yang (11): storage: Refactor the rng schema for storage pool auth storage: Support "username" for "chap" type "auth" storage: Add a struct for auth secret storage: Introduce XMLs to use secret object for pool auth storage: Output auth type before username storage: Support "chap" authentication for iscsi pool storage: Support to use secret object for iscsi chap "auth" storage: Update docs/formatsecret.html storage: Use the internal API to get the secret value instead storage: Improve the pool auth type parsing and formating Storage: Fix the indention of rbd test file docs/formatsecret.html.in | 10 +- docs/schemas/storagepool.rng | 51 +++--- src/conf/storage_conf.c | 179 ++++++++++++++------- src/conf/storage_conf.h | 28 +++- src/storage/storage_backend_iscsi.c | 114 ++++++++++++- src/storage/storage_backend_rbd.c | 13 +- .../storagepoolxml2xmlin/pool-iscsi-auth-login.xml | 17 ++ .../pool-iscsi-auth-secret.xml | 19 +++ .../pool-iscsi-auth-username.xml | 17 ++ tests/storagepoolxml2xmlin/pool-iscsi-auth.xml | 17 -- tests/storagepoolxml2xmlin/pool-rbd.xml | 2 +- .../pool-iscsi-auth-login.xml | 20 +++ .../pool-iscsi-auth-secret.xml | 22 +++ .../pool-iscsi-auth-username.xml | 20 +++ tests/storagepoolxml2xmlout/pool-iscsi-auth.xml | 20 --- .../pool-iscsi-vendor-product.xml | 2 +- tests/storagepoolxml2xmlout/pool-rbd.xml | 2 +- tests/storagepoolxml2xmltest.c | 3 +- 18 files changed, 424 insertions(+), 132 deletions(-) create mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-auth-login.xml create mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-auth-secret.xml create mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-auth-username.xml delete mode 100644 tests/storagepoolxml2xmlin/pool-iscsi-auth.xml create mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-auth-login.xml create mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-auth-secret.xml create mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-auth-username.xml delete mode 100644 tests/storagepoolxml2xmlout/pool-iscsi-auth.xml -- 1.8.1.4

2 30

[libvirt] [PATCH 00/19] Support for access control
by Daniel P. Berrange 20 Jun '13

20 Jun '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> This series (which depends on the Xen refactoring patches) adds support for access control checks on all APIs that run inside libvirtd. The first patch defines the basic objects which can be checked and the permissions associated with each object. In addition it provides the basic internal (pluggable) API for access control checks Later there are policykit and selinux drivers for the access control framework. Neither of these is currently optimal but they have basic functionality working To ensure that we don't forget access control checks when adding new APIs, we maintain metadata in the remote_protocol.x file against each method declaring what access control check must be done. There are actually two checks possible. The first check is against the object being used. The optional second check is against the objects being returned (if any). The latter is used to filter what can be seen when asking for a list of objects (eg 'virsh list' gets filtered) Again to ensure accurate checks, we automate the generation of methods for applying access control checks to each API. These helper methods are named to match the public API names. The last patch ensures that every method listed in the virXXXXDriverPtr tables has a call to an access control helper with the same name as the public API. And of course there are the patches which actually add the access control checks. Still todo - Not all Xen methods have access control checks yet. This causes the test case in the last patch to report failures - Have not wired up the checks for filtering the returned objects in any driver yet - The polkit driver is inefficient since it spawns pkcheck for each check. We need to talk to DBus directly since ACL checks will be very frequent and need to be lightweight - The SELinux driver is validating against the label of libvirtd. We need to validate against the label of the virDomainDefPtr security model or some equivalent for other objects. - Need to write a generic RBAC access control impl. It was hoped that new polkit would make this obsolete. Polkit is still unable to do access control checks for non-local users though eg it can't validate against SASL usernames or x509 certs. Daniel P. Berrange (19): Define basic internal API for access control Set conn->driver before running driver connectOpen method Setup default access control manager in libvirtd Add a policy kit access control driver Add an SELinux access control driver Add ACL annotations to all RPC messages Auto-generate helpers for checking access control rules Add ACL checks into the QEMU driver Add ACL checks into the LXC driver Add ACL checks into the UML driver Add ACL checks into the Xen driver Add ACL checks into the libxl driver Add ACL checks into the storage driver Add ACL checks into the network driver Add ACL checks into the interface driver Add ACL checks into the node device driver Add ACL checks into the nwfilter driver Add ACL checks into the secrets driver Add validation that all APIs contain ACL checks .gitignore | 10 + daemon/Makefile.am | 1 + daemon/libvirtd-config.c | 4 + daemon/libvirtd-config.h | 2 + daemon/libvirtd.aug | 1 + daemon/libvirtd.c | 27 ++ daemon/libvirtd.conf | 9 + daemon/test_libvirtd.aug.in | 4 + include/libvirt/virterror.h | 4 + m4/virt-compile-warnings.m4 | 1 + m4/virt-selinux.m4 | 2 + po/POTFILES.in | 3 + src/Makefile.am | 128 +++++- src/access/genpolkit.pl | 119 ++++++ src/access/viraccessdriver.h | 89 ++++ src/access/viraccessdrivernop.c | 118 ++++++ src/access/viraccessdrivernop.h | 28 ++ src/access/viraccessdriverpolkit.c | 399 ++++++++++++++++++ src/access/viraccessdriverpolkit.h | 28 ++ src/access/viraccessdriverselinux.c | 565 +++++++++++++++++++++++++ src/access/viraccessdriverselinux.h | 28 ++ src/access/viraccessdriverstack.c | 285 +++++++++++++ src/access/viraccessdriverstack.h | 32 ++ src/access/viraccessmanager.c | 352 ++++++++++++++++ src/access/viraccessmanager.h | 91 ++++ src/access/viraccessperm.c | 84 ++++ src/access/viraccessperm.h | 647 +++++++++++++++++++++++++++++ src/check-aclrules.pl | 144 +++++++ src/interface/interface_backend_netcf.c | 114 +++++ src/interface/interface_backend_udev.c | 85 +++- src/internal.h | 4 + src/libvirt.c | 11 +- src/libvirt_private.syms | 37 ++ src/libxl/libxl_driver.c | 187 ++++++++- src/locking/lock_protocol.x | 8 + src/lxc/lxc_driver.c | 219 +++++++++- src/network/bridge_driver.c | 61 +++ src/node_device/node_device_driver.c | 36 ++ src/nwfilter/nwfilter_driver.c | 26 ++ src/qemu/qemu_driver.c | 716 ++++++++++++++++++++++++++++---- src/remote/lxc_protocol.x | 1 + src/remote/qemu_protocol.x | 4 + src/remote/remote_protocol.x | 406 ++++++++++++++++++ src/rpc/gendispatch.pl | 212 +++++++++- src/secret/secret_driver.c | 31 ++ src/storage/storage_driver.c | 155 ++++++- src/uml/uml_driver.c | 174 +++++++- src/util/virerror.c | 8 + src/util/virlog.c | 3 +- src/util/virlog.h | 1 + src/xen/xen_driver.c | 217 +++++++++- 51 files changed, 5785 insertions(+), 136 deletions(-) create mode 100755 src/access/genpolkit.pl create mode 100644 src/access/viraccessdriver.h create mode 100644 src/access/viraccessdrivernop.c create mode 100644 src/access/viraccessdrivernop.h create mode 100644 src/access/viraccessdriverpolkit.c create mode 100644 src/access/viraccessdriverpolkit.h create mode 100644 src/access/viraccessdriverselinux.c create mode 100644 src/access/viraccessdriverselinux.h create mode 100644 src/access/viraccessdriverstack.c create mode 100644 src/access/viraccessdriverstack.h create mode 100644 src/access/viraccessmanager.c create mode 100644 src/access/viraccessmanager.h create mode 100644 src/access/viraccessperm.c create mode 100644 src/access/viraccessperm.h create mode 100644 src/check-aclrules.pl -- 1.8.1.4

3 21

[libvirt] [PATCH 0/2] Support setting the 'removable' flag for USB disks
by Fred A. Kemp 17 Jun '13

17 Jun '13

From: "Fred A. Kemp" <anonym(a)lavabit.com> For reference, my first attempt at a patch for this feature was: <1363682454-32012-1-git-send-email-anonym(a)lavabit.com> I believe this patch fixes all previous concerns raised that thread. However, note that I've only added the new capability for usb-storage.removable to the qemu help tests of qemu(-kvm) version 1.2.0, since that's what I had easily available to get the output of `-device usb-storage,?` from. Fred A. Kemp (2): qemu: Add capability flag for usb-storage qemu: Support setting the 'removable' flag for USB disks docs/formatdomain.html.in | 8 +++-- docs/schemas/domaincommon.rng | 8 +++++ src/conf/domain_conf.c | 31 ++++++++++++++++++-- src/conf/domain_conf.h | 1 + src/qemu/qemu_capabilities.c | 10 +++++++ src/qemu/qemu_capabilities.h | 2 ++ src/qemu/qemu_command.c | 23 +++++++++++++-- tests/qemuhelpdata/qemu-1.2.0-device | 11 +++++++ tests/qemuhelpdata/qemu-kvm-1.2.0-device | 11 +++++++ tests/qemuhelptest.c | 20 +++++++++---- .../qemuxml2argv-disk-usb-device-removable.args | 8 +++++ .../qemuxml2argv-disk-usb-device-removable.xml | 27 +++++++++++++++++ tests/qemuxml2argvtest.c | 6 +++- 13 files changed, 151 insertions(+), 15 deletions(-) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-usb-device-removable.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-disk-usb-device-removable.xml -- 1.7.10.4

2 3

[libvirt] [PATCH] Fix F_DUPFD_CLOEXEC operation args
by Daniel P. Berrange 12 Jun '13

12 Jun '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> The F_DUPFD_CLOEXEC operation with fcntl() expects a single int argument, specifying the minimum FD number for the newly dup'd file descriptor. We were not specifying that causing random stack data to be accessed as the FD number. Sometimes that worked, sometimes it didn't. Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- src/rpc/virnetsocket.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rpc/virnetsocket.c b/src/rpc/virnetsocket.c index e950d7f..dcf98b1 100644 --- a/src/rpc/virnetsocket.c +++ b/src/rpc/virnetsocket.c @@ -1055,7 +1055,7 @@ int virNetSocketDupFD(virNetSocketPtr sock, bool cloexec) int fd; if (cloexec) - fd = fcntl(sock->fd, F_DUPFD_CLOEXEC); + fd = fcntl(sock->fd, F_DUPFD_CLOEXEC, 0); else fd = dup(sock->fd); if (fd < 0) { -- 1.8.2.1

4 4

[libvirt] [PATCH] remote: fix dom->id after virDomainCreateWithFlags
by Marek Marczykowski 11 Jun '13

11 Jun '13

The same issue as (already fixed) in virDomainCreate - REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS doesn't return new domain ID, only -1 on error or 0 on success. Besides this one fix it is more general problem - local domain object ID can desynchronize with the real one, for example in case of another client creates/destroys domain in the meantime. Perhaps virDomainGetID should be called remotely (with all performance implications...)? Or some event-based notification used? Signed-off-by: Marek Marczykowski <marmarek(a)invisiblethingslab.com> --- src/remote/remote_driver.c | 40 ++++++++++++++++++++++++++++++++++++++++ src/remote/remote_protocol.x | 2 +- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c index f66304c..0bbfc22 100644 --- a/src/remote/remote_driver.c +++ b/src/remote/remote_driver.c @@ -2423,6 +2423,46 @@ done: return rv; } +static int +remoteDomainCreateWithFlags(virDomainPtr dom, unsigned int flags) +{ + int rv = -1; + struct private_data *priv = dom->conn->privateData; + remote_domain_create_with_flags_args args; + remote_domain_lookup_by_uuid_args args2; + remote_domain_lookup_by_uuid_ret ret2; + + remoteDriverLock(priv); + + make_nonnull_domain(&args.dom, dom); + args.flags = flags; + + if (call(dom->conn, priv, 0, REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS, + (xdrproc_t)xdr_remote_domain_create_with_flags_args, (char *)&args, + (xdrproc_t)xdr_void, (char *)NULL) == -1) { + goto done; + } + + /* Need to do a lookup figure out ID of newly started guest, because + * bug in design of REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS means we aren't getting + * it returned. + */ + memcpy(args2.uuid, dom->uuid, VIR_UUID_BUFLEN); + memset(&ret2, 0, sizeof(ret2)); + if (call(dom->conn, priv, 0, REMOTE_PROC_DOMAIN_LOOKUP_BY_UUID, + (xdrproc_t) xdr_remote_domain_lookup_by_uuid_args, (char *) &args2, + (xdrproc_t) xdr_remote_domain_lookup_by_uuid_ret, (char *) &ret2) == -1) + goto done; + + dom->id = ret2.dom.id; + xdr_free((xdrproc_t) &xdr_remote_domain_lookup_by_uuid_ret, (char *) &ret2); + rv = 0; + +done: + remoteDriverUnlock(priv); + return rv; +} + static char * remoteDomainGetSchedulerType(virDomainPtr domain, int *nparams) { diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x index 512ba2e..d2e0175 100644 --- a/src/remote/remote_protocol.x +++ b/src/remote/remote_protocol.x @@ -3875,7 +3875,7 @@ enum remote_procedure { REMOTE_PROC_DOMAIN_EVENT_IO_ERROR_REASON = 195, /** - * @generate: both + * @generate: server */ REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS = 196, -- 1.8.1.4

3 2

[libvirt] [PATCH 0/2] Resolve issues seen with schedinfo
by John Ferlan 08 Jun '13

08 Jun '13

These patches resolve an issue seen using 'virsh schedinfo <domain>' on a non running domain that have been present since 1.0.4 as a result of the cgroup infrastructure changes: https://www.redhat.com/archives/libvir-list/2013-April/msg00783.html The exact commit id that caused the issue is listed in each of the commit messages. I used git bisect to determine, although it was tricky because the TPM changes were made around the same time and required commit '8b934a5c' to be applied in order to actually see domains on my host. Prior to the changes the "CFS Bandwidth" data provided by qemuGetCpuBWStatus() and lxcGetCpuBWStatus() would be obtained as a result of the driver cgroup being mounted. Now it relies on the domain cgroup being mounted which only occurs once the domain is active. This issue also affects the libvirt-cim code in how it defines QEMU domains. Fortunately it only looks for the "cpu_shares" value. A "downside" to the end result of these changes is that for a non-running domain it becomes impossible to obtain the vcpu_period, vcpu_quota, emulator_period, and emulator_quota values. All that can be obtained is the 'cpu_shares" value. As an option, I did consider adding the following to either [qemu|lxc]DomainSchedulerGetType() and DomainSchedulerParameter[Flags]() or just [qemu|lxc]GetCpuBWStatus() APIs in order to get the "host" values if they existed (since they are eventually copied into the domain cgroup): if (cgroup == NULL) { rc = virCgroupNewSelf(&hostgrp); if (rc < 0) { virReportSystemError(-rc, "%s", _("Unable to get schedinfo host cgroup")); goto cleanup; } cgroup = hostgrp; } However, I wasn't sure going that route was desired and figured that I'd use the code review opportunity to get the answer. Furthermore, it wasn't clear that the vcpu_* and emulator_* values made sense for a non running domain. Also, the virReportSystemError may need to change to a VIR_INFO since I believe it would be triggered if cgroups weren't mounted on the system. Another option would be to just add the above code to the GetType() APIs and then ignore the 'cpu_bw_status' for the VIR_DOMAIN_AFFECT_CONFIG path in the ParametersFlags() APIs thus returning all 5 datapoints set to whatever the persistentDef had defined. Since this has been present since 1.0.4 and no one has complained so far, I don't think it's critical for 1.0.6. I suspect the change would need to into the maint trees though. It might be nice to get DanB's opinion on this too. John Ferlan (2): qemu: Resolve issue with virsh schedinfo for non running domain lxc: Resolve issue with virsh schedinfo for non running domain src/lxc/lxc_driver.c | 9 ++++++++- src/qemu/qemu_driver.c | 11 ++++++++++- 2 files changed, 18 insertions(+), 2 deletions(-) -- 1.8.1.4

3 8