September 2013 - Devel - libvirt List Archives

[libvirt] memory pinning problem
by David Weber 19 Sep '13

19 Sep '13

Hi, we try to use vcpu pinning on a 2 socket server with Intel Xeon E5620 cpus, HT enabled and 2*6*16GiB Ram but experience problems if we try to start a guest on the second socket: error: Failed to start domain test error: internal error: process exited while connecting to monitor: kvm_init_vcpu failed: Cannot allocate memory Libvirt version 1.1.1 Linux 3.11-rc7 Because I coudn't find any other service which allowed a 7M file upload, I put the log file and everything else which could perhabs be relevant into a github repository: https://github.com/David-Weber/vcpu-pinning When we try to start a guest on the first node it runs fine: <vcpu placement='static' cpuset='0-3,8-11'>4</vcpu> <numatune> <memory mode='strict' nodeset='0'/> </numatune> Starting it on the second node fails <vcpu placement='static' cpuset='4-7,12-15'>4</vcpu> <numatune> <memory mode='strict' nodeset='1'/> </numatune> Even more strange, starting it with the CPUs of the second node and the memory of the first node works: <vcpu placement='static' cpuset='4-7,12-15'>4</vcpu> <numatune> <memory mode='strict' nodeset='0'/> </numatune> The log file contains these three cases. Using the placement='auto' parameter leads to the same problem. If numad return the second node, the guest won't start. Is this a configuration, a libvirt or a cgroup problem? :) Cheers, David

2 7

[libvirt] Auto-assignment of USB devices and auto-creation of USB hubs
by Jamie Bainbridge 19 Sep '13

19 Sep '13

Hello, I've been helping a customer out this week with assigning specific devices to specific USB controllers inside his KVM VMs. He's got USB1 devices which need a UHCI controller (because of rubbishy Windows device drivers) and USB2 devices which need a EHCI controller ("speed mismatch" on a UHCI hub). I've suggested config similar to this: <controller type='usb' index='0' model='ehci'> </controller> <controller type='usb' index='1' model='piix3-uhci'> </controller> <input type='tablet' bus='usb'/> <hub type='usb'> <address type='usb' bus='1' port='2'/> </hub> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0xAAAA'/> <product id='0xAAAA'/> </source> <address type='usb' bus='1' port='2.1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0xBBBB'/> <product id='0xBBBB'/> </source> <address type='usb' bus='1' port='2.2'/> </hostdev> and I explained the concept to him with diagrams like these: UHCI --+-- Tablet (port 1) +-- HUB (port 2) --. +-- Device (port 2.1) +-- Device (port 2.2) UHCI --+-- Tablet (port 1) +-- HUB (port 2) --. +-- HUB (port 2.1) --. | +-- Device (port 2.1.1) | +-- Device (port 2.1.2) | +-- HUB (port 2.2) --. +-- Device (port 2.2.1) (then imagine extending this example out to also include EHCI controller and devices) He's very happy so far and I've just been working it all out through trial and error and reading the libvirt domain specification. My observation is that you can either: a) let libvirt plug everything into the UHCI controller, regardless of what other controllers are there and what order they appear in or b) find out how many ports a controller/hub has through trial and error, manually create USB hubs, and map every single device and port out manually with no middle ground in between. I think it would be way better if users could just assign a USB hostdev to a controller, then let libvirt do all the heavy lifting in regards to hubs and ports. For example, : <controller type='usb' index='0' model='ehci'> </controller> <controller type='usb' index='1' model='piix3-uhci'> </controller> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x0529'/> <product id='0x0001'/> </source> <address type='usb' bus='1'/> </hostdev> <hostdev mode='subsystem' type='usb' managed='yes'> <source> <vendor id='0x3923'/> <product id='0x709b'/> </source> <address type='usb' bus='1'/> </hostdev> I can see it's already been discussed that we couldn't just implement "port=auto" because we need migration stability: http://www.redhat.com/archives/libvir-list/2011-August/msg00816.html but much like how you add: <controller type='usb' index='0' model='ehci'> </controller> then save the XML, and go "virsh edit" and the controller automagically has a PCI address assigned, we could enter this into the XML: <hostdev mode='subsystem' type='usb' managed='yes'> <source> whatever </source> <address type='usb' bus='1'/> </hostdev> then save the XML, and "virsh edit" again, and all the port/hub stuff would already be there pre-made as required. Any thoughts? Is this worth logging an RFE for? Cheers, Jamie

3 2

Re: [libvirt] [virt-tools-list] [Spice-devel] More on virt-viewer for windows
by Eric Blake 19 Sep '13

19 Sep '13

[adding libvir-list, for some cross-compiling development hints] On 09/17/2013 11:57 AM, Fernando Lozano wrote: > Yes, the libvirt comes grom 0.10.2. I'm running the latest windows > binaries provided by spice-space.org: > > C:\Program Files\VirtViewer\bin>virsh -V > Virsh command line tool of libvirt 0.10.2 Can someone from the Spice community chime in? Why is spice-space.org shipping a Fedora 18 build of libvirt (0.10.2.x) rather than Fedora 19 (1.0.5.x)? Who does the builds, and how often are they updated? >> Process Monitor is only useful if you make system calls; but libvirt is >> choking even before attempting the system calls because mingw is just >> such a hostile programming environment to programs that assume POSIX. > > That's precisely mingw advantage over cigwin: mingw binaries are native > windows binaries, using native windows semantics, not unix emulation. > They provide a better experience for windows users. Welcome to the > wonderful world of cross-platform developent! ;-) Actually, on my personal to-do list is to get a build of virsh working and available from cygwin.com, rather than requiring people to self-build it for cygwin. And since cygwin supports cross-compilation to mingw, just as Fedora does, but with the added advantage of also being able to run mingw binaries without having to worry about bugs in the wine emulation layer, it's actually a bit nicer for getting the best of both worlds when doing cross-compiling (posix development, native testing). If I can ever find the free time... > > >> Gnulib has helped a lot, and often times, it is just a matter of someone >> running under gdb to see where an assumption went wrong to make a quick >> patch to fix an issue. Where it gets tricky is that it is hard to find >> developers willing to do volunteer work on issues for a platform where >> you typically have to pay money before you can even use it. Also, the >> fact that you are using a pre-built version of a relatively old libvirt, >> instead of building your own from the latest sources, makes it hard to >> know what OTHER issues may have been fixed in the meantime (when given a >> choice, developers prefer to debug issues in the latest source, rather >> than trying to figure out which patches to backport to older branches). > I understand that, but I'm trying to be useful as a (windows) tester. If > I could I'd try to help as a developer. cross-compiling is not for the > faint of heart, and learning the first steps require a significant > investment in time. :-( > > Is there a how-to I can follow to generate binaries from the latest > sources? I do have Linux expertize, I use fedora on my personal > computer, but as C developer I can only run "configure; make; sudo make > install". All I've done so far is test cross-compilation from Fedora to mingw (I haven't tested the resulting binaries, only that the build completes without error). I suppose I could try to host the just-built binaries on a site for you to download; let me see what I can figure out. Meanwhile, I used './autobuild.sh' as my guide; which means I more or less did: $ sudo yum install \ mingw64-binutils \ mingw64-cpp \ mingw64-crt \ mingw64-curl \ mingw64-dbus \ mingw64-filesystem \ mingw64-gcc \ mingw64-gcc-c++ \ mingw64-gettext \ mingw64-gmp \ mingw64-gnutls \ mingw64-headers \ mingw64-libgcrypt \ mingw64-libgpg-error \ mingw64-libidn \ mingw64-libssh2 \ mingw64-libtasn1 \ mingw64-libxml2 \ mingw64-nettle \ mingw64-openssl \ mingw64-p11-kit \ mingw64-pkg-config \ mingw64-portablexdr \ mingw64-termcap \ mingw64-wine-gecko \ mingw64-win-iconv \ mingw64-zlib $ git clone git://libvirt.org/libvirt.git $ cd libvirt $ ./autogen.sh --help # see note [1] $ mkdir build-mingw $ cd build-mingw $ ../configure \ PKG_CONFIG_LIBDIR="/usr/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig:/usr/x86_64-w64-mingw32/sys-root/mingw/share/pkgconfig" \ PKG_CONFIG_PATH="$AUTOBUILD_INSTALL_ROOT/x86_64-w64-mingw32/sys-root/mingw/lib/pkgconfig" \ CC="x86_64-w64-mingw32-gcc" \ --build=$(uname -m)-w64-linux \ --host=x86_64-w64-mingw32 \ --prefix="$AUTOBUILD_INSTALL_ROOT/x86_64-w64-mingw32/sys-root/mingw" \ --without-libvirtd \ --without-python $ make [1] Yeah, we probably ought to have a more convenient way to bootstrap libvirt.git for VPATH builds without having to rely on the hack that options unrecognized by ./autogen.sh are passed on to ./configure, and that ./configure --help is the easiest way to do avoid polluting the top-level tree in a way that would interfere with VPATH builds. I use a VPATH build for my mingw cross-compilation in part so I can test both 32-bit and 64-bit binaries in parallel directories. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

6 8

[libvirt] [v0.9.12-maint 0/5] building 0.9.12 on newer systems
by Eric Blake 19 Sep '13

19 Sep '13

I'm pushing these backports under the build-breaker rule; they were all necessary for me to complete 'make check' on this old branch from within Fedora 19. It will set the stage for the CVE patches in another series. Daniel P. Berrange (1): Fix TLS tests with gnutls 3 Eric Blake (3): build: allow building with newer glibc-headers and -O0 build: more workarounds for if_bridge.h build: avoid confusing make with raw name 'undefine' Jiri Denemark (1): virsh: Fix POD syntax configure.ac | 20 +++++++++++++++++--- m4/virt-compile-warnings.m4 | 8 ++++++-- src/util/virnetdevbridge.c | 18 ++++++++++++++++++ tests/Makefile.am | 10 ++++++---- tests/virnettlscontexttest.c | 25 +++++++++++++++---------- tests/{undefine => virsh-undefine} | 0 tools/virsh.pod | 4 ++-- 7 files changed, 64 insertions(+), 21 deletions(-) rename tests/{undefine => virsh-undefine} (100%) -- 1.8.3.1

1 5

[libvirt] Remove more hardcoded paths from libvirtd --help
by Christophe Fergeau 18 Sep '13

18 Sep '13

In my previous 2 patches, I used constants defined in remote_driver.h to build privileged libvirtd --help output. However, there are a few more constants in this header that we can use to build the --help output, this series addresses that. Christophe

2 4

[libvirt] [PATCH] docs: fix virEventAddHandle return details
by Jonathan Lebon 18 Sep '13

18 Sep '13

In commit 6d41cb8, the interface for virEventAddHandleFunc was changed. This patch updates the documentation for virEventAddHandle to reflect the new significance of the return value. Also, both functions now mention -1 for failure. --- include/libvirt/libvirt.h.in | 4 ++-- src/util/virevent.c | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index a47e33c..7944dfb 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -3643,8 +3643,8 @@ typedef void (*virEventHandleCallback)(int watch, int fd, int events, void *opaq * If 'ff' callbacks are invoked directly from the virEventRemoveHandleFunc * they will likely deadlock in libvirt. * - * Returns a handle watch number to be used for updating - * and unregistering for events + * Returns -1 if the file handle cannot be registered, otherwise a handle + * watch number to be used for updating and unregistering for events */ typedef int (*virEventAddHandleFunc)(int fd, int event, virEventHandleCallback cb, diff --git a/src/util/virevent.c b/src/util/virevent.c index 886cb6d..fde29a2 100644 --- a/src/util/virevent.c +++ b/src/util/virevent.c @@ -48,7 +48,8 @@ static virEventRemoveTimeoutFunc removeTimeoutImpl = NULL; * * Register a callback for monitoring file handle events. * - * Returns -1 if the file handle cannot be registered, 0 upon success + * Returns -1 if the file handle cannot be registered, otherwise a handle + * watch number to be used for updating and unregistering for events */ int virEventAddHandle(int fd, -- 1.8.3.1

2 1

[libvirt] [PATCH] Fix crash in remoteDispatchDomainMemoryStats (CVE-2013-4296)
by Eric Blake 18 Sep '13

18 Sep '13

From: "Daniel P. Berrange" <berrange(a)redhat.com> The 'stats' variable was not initialized to NULL, so if some early validation of the RPC call fails, it is possible to jump to the 'cleanup' label and VIR_FREE an uninitialized pointer. This is a security flaw, since the API can be called from a readonly connection which can trigger the validation checks. This was introduced in release v0.9.1 onwards by commit 158ba8730e44b7dd07a21ab90499996c5dec080a Author: Daniel P. Berrange <berrange(a)redhat.com> Date: Wed Apr 13 16:21:35 2011 +0100 Merge all returns paths from dispatcher into single path Signed-off-by: Daniel P. Berrange <berrange(a)redhat.com> --- Approved on the security list, and pushing on Daniel's behalf. I'm now working on backporting this to the various v*-maint branches. daemon/remote.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemon/remote.c b/daemon/remote.c index 6b082cf..3ac528c 100644 --- a/daemon/remote.c +++ b/daemon/remote.c @@ -1146,7 +1146,7 @@ remoteDispatchDomainMemoryStats(virNetServerPtr server ATTRIBUTE_UNUSED, remote_domain_memory_stats_ret *ret) { virDomainPtr dom = NULL; - struct _virDomainMemoryStat *stats; + struct _virDomainMemoryStat *stats = NULL; int nr_stats; size_t i; int rv = -1; -- 1.8.3.1

1 0

[libvirt] [PATCH 0/8] Normalization API
by Michal Privoznik 18 Sep '13

18 Sep '13

A while ago I've invented this vshCompleteXMLFromDomain() function to increase device-detach intelligence. Basically, it took user's XML and tried to find matching device in domain's XML. However, it was kind of buggy - finding the matching device uses string comparison. This works on text values. It doesn't on integer ones (who would expect that). So prior the lookup process, we need to normalize the integer values (and whole XML). However, this can't be done on the client side, since he has no knowledge which values are integer and which are not. Therefore we need a new API. Michal Privoznik (8): domain_conf: Introduce virDomainDeviceDefFormat Introduce new virDomainNormalizeXML API remote_driver: Implement virDomainNormalizeXML virsh: Expose virDomainNormalizeXML qemu: Implement vimDomainNormalizeXML domain_conf: Move MAC generation to post parse callback virDomainDeviceDefParse: Make PostParse callback call optional virsh: Resurrect vshCompleteXMLFromDomain daemon/remote.c | 50 ++++++++ include/libvirt/libvirt.h.in | 5 + python/generator.py | 1 + python/libvirt-override-api.xml | 7 ++ python/libvirt-override.c | 30 +++++ src/conf/domain_conf.c | 180 ++++++++++++++++------------- src/conf/domain_conf.h | 7 +- src/driver.h | 7 ++ src/libvirt.c | 51 ++++++++ src/libvirt_private.syms | 1 + src/libvirt_public.syms | 5 + src/libxl/libxl_driver.c | 12 +- src/lxc/lxc_driver.c | 6 +- src/openvz/openvz_conf.c | 1 + src/openvz/openvz_driver.c | 12 +- src/phyp/phyp_driver.c | 2 +- src/qemu/qemu_command.c | 7 +- src/qemu/qemu_driver.c | 51 +++++++- src/remote/remote_driver.c | 40 +++++++ src/remote/remote_protocol.x | 20 +++- src/uml/uml_driver.c | 4 +- src/vbox/vbox_tmpl.c | 5 +- src/vmx/vmx.c | 2 + src/xen/xend_internal.c | 6 +- src/xen/xm_internal.c | 8 +- src/xenapi/xenapi_driver.c | 1 + src/xenxs/xen_sxpr.c | 1 + src/xenxs/xen_xm.c | 1 + tests/openvzutilstest.c | 1 - tests/qemuhotplugtest.c | 3 +- tools/virsh-domain.c | 249 +++++++++++++++++++++++++++++++++++++++- tools/virsh.pod | 8 ++ 32 files changed, 666 insertions(+), 118 deletions(-) -- 1.8.1.5

4 21

[libvirt] [PATCH] doc: fix XML for the RNG device example
by Giuseppe Scrivano 18 Sep '13

18 Sep '13

Add a missing '/' to close the "source" element. Signed-off-by: Giuseppe Scrivano <gscrivan(a)redhat.com> --- A minor detail that I have noticed while I copy-pasting the example. docs/formatdomain.html.in | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index a927643..3689399 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -4903,8 +4903,8 @@ qemu-kvm -net nic,model=? /dev/null <backend model='random'>/dev/random</backend>  <backend model='egd' type='udp'> - <source mode='bind' service='1234'> - <source mode='connect' host='1.2.3.4' service='1234'> + <source mode='bind' service='1234'/> + <source mode='connect' host='1.2.3.4' service='1234'/> </backend> </rng> </devices> -- 1.8.3.1

2 1

[libvirt] Integrating LiSA into libvirt
by Andreea Hodea 18 Sep '13

18 Sep '13

My name is Andreea and I am a member of LiSA team. LiSA stands for Linux Switching Appliance and offers a software switch that switches packets at Layer 2 and Layer 3. It is an open source project under development, you can find more about it from lisa.mindbit.ro or from https://github.com/lisa-project/. One of the development directions of LiSA is to integrate it into libvirt. First I'll state the motivation for such an integration and then I will describe a use case. The main reasons for extending libvirt network component with LiSA are related to the fact that different hypervisors offer different switching engines with different capabilities. By integrating LiSA into libvirt, the virtual machines would connect to LiSA switch ports as they were linked through a physical switch. Hence, with LiSA, the switching engine would offer the same switching capabilities independent of hypervisor. Moreover, if I understand correctly, in libvirt switching configurations are possible either through a set of commands in virsh or directly within a XML file. One of LiSA components is swcli that is a CLI for configuring and retrieving configurations of the software switch. Through swcli, the switching configurations (including VLANs, trunk or access mode interfaces etc.) are made through a CISCO-IOS like set of commands. I think this is a nice-to-have because it would separate complex network configurations from NIC configuration. That is, through libvirt the machines would just connect to the switch and other switching configurations, simple or complex, would be made from swcli. In other words, the XML file would remain simple and readable while the user would still be able to make switching specific configurations. In conclusion, LiSA and libvirt would make a great team for building a complex networking model, without complicating the setup of a virtual network. Scenario of where LiSA fits into libvirt =========================== I thought at the following scenario: two virtual machines linked together through a LiSA switch, i.e. the LiSA switch being the physical machine running LiSA. We, LiSA team, started by studying the virt-manager interface and figured out that an entry in the "Source device" drop down list when configuring a domain NIC should suffice our goal for the moment. When applying these changes, the XML configuration file should contain an interface tag of type "lisa-switch" or something similar. Therefore, a patch to libvirt for integrating LiSA would contain: - the implementation of two functions - virNetDevLisaAddPort - virNetDevLisaRemovePort - and a new interface type (XML level) for network configuration. I would like to have your feedback on the ideas above. Thanks, Andreea

4 4