- Devel - libvirt List Archives

[Libvir] Xen physical host statistics
by Mindaugas Kiznis 10 Aug '07

10 Aug '07

Hello, - I am trying to make an application, which would connect to a remote xen host and fetch some information about total (physical) CPU usage, total memory usage, network and disk I/O. I have managed to do a remote statistics grabber for CPU and memory usage using libvirt. I couldn't find any source of information about libvirt ability to return I/O numbers. Also haven't found any other solution yet. Any suggestions? Thank you in advance! Best regards, Mindaugas Kiznis

3 4

[Libvir] PATCH: UUID code cleanup
by Daniel P. Berrange 10 Aug '07

10 Aug '07

Browsing the source I noticed we have many differents constants for UUID length, many ways of turning a UUID into a string, 2 ways of turning a string into a UUID (one with some scary signed <-> unsigned casting). This is just a result of the way the code evolved & I figured it could do with cleaning up a little. So I added a virUUIDFormat(const unsigned char uuid, char *uuidstr) method to the uuid.c, removed all the other equivalent code throughout and kiled all the duplicate constants. So now everything UUID related is in the one place uuid.c, while the constants are in libvirt/libvirt.h Its a surprisingly big patch, but it should not have any functional change with one exception. The XenD driver used to generate XML <uuid>...</uuid> tags without any '-' embedded in them. All other places use the '-' and technically the RFC requires them. Our parser is flexible enough to cope with, or without '-'. diffstat ~/libvirt-uuid-cleanup.patch proxy/Makefile.am | 2 - qemud/internal.h | 2 - src/libvirt.c | 34 +++------------------------- src/openvz_conf.c | 14 +++++------ src/openvz_conf.h | 3 -- src/qemu_conf.c | 20 ++++++---------- src/qemu_conf.h | 5 +--- src/test.c | 18 +++++---------- src/uuid.c | 57 +++++++++++++++++++++++++++++++++-------------- src/uuid.h | 13 ++++------ src/xend_internal.c | 62 +++++++++++++++++++--------------------------------- src/xm_internal.c | 31 +++++--------------------- src/xml.c | 61 --------------------------------------------------- src/xml.h | 1 14 files changed, 104 insertions(+), 219 deletions(-) Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

3 3

[Libvir] ANNOUNCE: Virt-top - a top-like utility for displaying virtualization stats
by Richard W.M. Jones 10 Aug '07

10 Aug '07

I'm pleased to announce the first release of virt-top, which is a top-like utility for displaying virtualization stats. It aims to look and feel very much like regular 'top', so as to be as familiar as possible for systems administrators. You can also use it as a pleasant replacement for xentop. It uses libvirt, so can display stats across a variety of different hypervisors and virtualization systems (not just Xen, although that is where the testing has gone so far). http://et.redhat.com/~rjones/virt-top/ The license is a combination of LGPL (for the library) and GPL (for the virt-top program). Current status -------------- You can view domains and use familiar keys like 'P'/'M'/... to sort by processor/memory/..., and 'd'/'s' to set the delay between updates. Also some common top command-line options are implemented. The man page is here: http://et.redhat.com/~rjones/virt-top/virt-top.txt There are a variety of source and binary RPMs available for Fedora users. I don't yet have a working Debian/Ubuntu package, but will have a go at making one tomorrow. The next thing I'll be working on is showing virtual and physical CPU usage of guests. After that I'm hoping to discuss extensions to libvirt to make other interesting statistics available to virt-top, in particular disk and network I/O stats. There are a few data collection artifacts which need to be investigated. In particular, %CPU sometimes goes over 100%. Obviously accurate data collection is an important goal for this tool. Memory usage is good: typical 'RES' (in regular top) for virt-top is just under 2.5 MB, and I've had it running for hours at a time without memory usage increasing, which seems to indicate that there aren't any major memory leaks. Development ----------- The program is currently very small: exactly 500 lines of code! If you want to dive in and send me patches they are most welcome, but remember that I'm trying to make this utility act as much like 'top' as possible, so if 'top' does it in a particular way, then I'd prefer virt-top to do the same thing. (May not apply to obscure top functionality, if top does something silly). If you're a systems administrator, not a programmer, and you'd like virt-top to have some particular feature, then please let me know, and I will be happy to review it and code it for you. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

4 10

[Libvir] libvirt daemon UNIX socket auth with PolicyKit
by Daniel P. Berrange 09 Aug '07

09 Aug '07

Currently our authentication model for local connections is using the basic UNIX file permissions, possibly with a setuid helper (in Xen case only). It can be summarized as - If app using libvirt is running as root => full access - Else => read only access The latter is enforced by fact that in Xen case libvirt_proxy only has impl for a handful of read only APIs, or in non-Xen case that the UNIX domain socket for the daemon /var/run/libvirt/libvirt-sock is mode 0700, while /var/run/libvirt/libvirt-sock-ro is 0777 & the daemon enforces based on which socket the client connects to. This is good because it allows non-root to at least monitor guest state while requiring root authentication for actually changing state. This is bad because it requires any app which wants to change state to run as root. ie we are required to launch virt-manager as root to gain ability to manage local guests. Problem with this include: - running the entire PyGTK & GTK & X codebase as root is undesirable - no integration with the DBus desktop session (gnome-vfs integartion) - no integration with the GNOME keyring (for VNC server passwords) - redundant (&dangerous) if all you want to do is manage remote libvirt hosts In summary what I really need for virt-manager is - Always run as non-root - Authenticate for local guest management (ie read+write) UNIX domain sockets already provide a way for each end to identify the PID and UID of the other end. This enables the libvirt daemon to determine the identity of the application on the other end. With this information the daemon merely needs to check this identity against some access control policy rules. Where to get/define these rules though ? Enter PolicyKit. http://lists.freedesktop.org/archives/hal/2006-March/004770.html http://lists.freedesktop.org/archives/hal/2007-June/008815.html PolicyKit is a freedesktop.org project to replace all the distro specific 'do admin stuff as root' hacks like consolehelper[1], sudo, gksudo with a standard mechanism. Most importantly this mechanism does not require that the application in question run as root, or ever gain root privileges. In addition this is completely configurable by the administrator in a application independant manner. How would the daemon use PolicyKit for authenticating clients on UNIX domain sockets - libvirtd defines two actions it can check called 'libvirt-local-monitor (read only monitoring of state), and 'libvirt-local-manage' (full read-write management). - Both libvirt-sock and libvirt-sock-ro are mode 0777 (ie all users) - A connection arrives on the UNIX domain socket either libvirt-sock-ro or libvirt-sock - libvirtd use SO_PEERCRED to get the PID of the client - If the connection is on libvirt-sock-ro 'action' is 'libvirt-local-monitor' Else if connection is on libvirt-sock 'action' is 'libvirt-local-manage' - Libvirt asks policy kit if PID is allowed to perform the 'action' -> Yes, connection proceeds -> No, connection is dropped NB, having two separate UNIX domain sockets is strictly speaking redundant now - the daemon could simply check each 'action' in turn - but keeping the separate sockets simplifies the code, because it minimises the diffs when PolicyKit is not enabled by the 'configure' script. How is policy determined by PolicyKit ? - libvirt RPM provides /usr/share/PolicyKit/policy/libvirt.policy - This file defines the two actions it supports 'libvirt-local-monitor' and 'libvirt-local-manage'. - For each action we define a default policy for an active desktop session, and an inactive desktop session. The sample policy is to allow 'libvirt-local-monitor' for any local desktop session, but only allow 'libvirt-local-manage' for the active session with the additional requirement that the user authenticate with PolicyKit using the root password. This basically replicates the existing security semantics, without requiring that we run virt-manager itself as root. This accomplishes the core goal. There are other benefits to PolicyKit though - the administrator can override the application provided default Policy. For example, could add a rule that says "if the user fred is logged into the local X session, allow them to do 'libvirt-local-manage' without any further prompts for root password" Since PolicyKit is a general framework, that rule could be generalized beyond just virt-manager "if the user fred is logged into the local X session, allow them to do any defined action without any further prompts for root password" Or going the other way the administrator can lock things down requiring a root password even for read only monitoring of VMs. Or a halfway house of requiring the user's own password (sudo equiv, instead of su equiva). If course this doesn't just apply to virt-manager either. This transparently 'just works' for virsh too & any other apps using libvirt - eg if policy grants 'libvirt-local-manage' then you can do full management whether virsh is root or not. This all talks about enforcement of policy from the daemon side. There is one outstanding question - if the policy so requires, where/how do we ask the user to enter their credentials. PolicyKit provides two mechanisms for this: - A command line tool polkit-grant which uses pam to authenticate you as root, or using your own user password. If successful you will have been granted the permission to perform that action for the remainder of the session. - A DBus service for triggering display of some UI for authenticating. There is a GNOME impl of this service & likely a KDE impl too. eg with virsh start off without any priveleges $ virsh --connect qemu:///system net-list libvir: Remote error : Connection reset by peer error: failed to connect to the hypervisor error: no valid connection $ polkit-grant --action libvirtd-local-manage2 Authentication is required. Password: Keep this privilege for the session? [no/session]? $ ./src/virsh --connect qemu:///system net-list --all Name State Autostart ----------------------------------------- default active yes One option is to have the libvirt library spawn polkit-grant if not finding any $DISPLAY, and call the DBus service (for UI prompt) if finding any. The other option is to punt this code upto the application - eg virt-manager always call the DBus service, while virsh always call polkit-grant. Finally, PolicyKit is new in Fedora 8 if you wish to try this out. The configure script is setup so that it automagically detects presence of PolicyKit and only turns it on at compile time if its available. The fallback is to continue current behaviour of restricting based on the socket permissions. http://fedoraproject.org/wiki/Releases/FeaturePolicyKit Finally, finally, if we enable PolicyKit support, then we should disable the Xen specific libvirt_proxy setuid binary, since it would subvert the policy for libvirt-local-monitor. IMHO we should just kill the setuid binary altogether, since we already require that the libvirt daemon be running for the networking APIs & this provides read-only access already which works for non-Xen drivers too. Attaching the patch implementing the proof of concept $ diffstat ~/libvirt-policykit.patch configure.in | 60 ++++++++++++++++++-- qemud/Makefile.am | 12 +++- qemud/internal.h | 23 ++++++- qemud/libvirtd.policy | 41 ++++++++++++++ qemud/qemud.c | 145 ++++++++++++++++++++++++++++++++++++++++++++------ qemud/remote.c | 6 +- 6 files changed, 258 insertions(+), 29 deletions(-) Oh, finally, finally, finally. PolicyKit is pretty new so I'd understand if there's some resistance to including it in an official libvirt release at this point. I think the patch is small enough that we could carry it as add-on in the Fedora RPMs only to start with untill its proved to be maintainable long term - I really need this for Fedora 8 to make virt-manager work sanely - ie not need to run as root. Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

4 9

[Libvir] [PATCH] A variety of fixes for Debian, old versions of GnuTLS
by Richard W.M. Jones 08 Aug '07

08 Aug '07

Attached is a patch to get libvirt working on Debian/stable again. (1) Add ./configure option --disable-stack-protector. The stack protector feature is still enabled by default, but it causes no end of problems when compiling on Debian systems, apparently because of bugs in their gcc. So Debian users would be well advised to try disabling it if they have problems building. (2) Adds support for GnuTLS 1.0.x by detecting this version and enabling some simple compatibility macros. (3) In qemudDispatchClientEvent, a prototype shadowed a global function name. A simple name change to a parameter fixes it. (4) Debian (and likely non-Linux systems also) don't have the SIOCBRADDBR and related ioctls. If these are missing, then I've changed this to runtime errors. This means that these systems won't be able to start the network daemon, but other (non-network) features of libvirt continue to work. (5) Removed a warning in src/qemu_conf.c. Rich. PS. You need to drop the file gnutls_1_0_compat.h into src/ subdirectory. Sorry this file isn't in the proper patch but I don't have write access to CVS from the testing machine. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

3 3

[Libvir] [PATCH] Fix libvirtd --config / -f option
by Richard W.M. Jones 08 Aug '07

08 Aug '07

As well as fixing the --config/-f option for libvirtd, this also changes an abort() into a meaningful error message. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

3 4

[Libvir] ANNOUNCE: OCaml bindings 0.3.1.0 released
by Richard W.M. Jones 02 Aug '07

02 Aug '07

http://et.redhat.com/~rjones/ocaml-libvirt/ This removes use of the dangerous vir{Domain,Network}GetConnect functions and makes the relationship between domains, networks and connections explicit to the garbage collector. I've also added support for domain migration. mlvirtmanager displays the hostname when connected to multiple remote hosts (see screenshot attached). Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

1 0

[Libvir] [PATCH] check file format in virsh attach/detach-device
by Masayuki Sunou 02 Aug '07

02 Aug '07

Hi virsh attach/detach-devich does not check file format now. This patch checks config file format is XML or not. (in virsh attach/detach-device) If file format is not XML, just return the error with message. Signed-off-by: Masayuki Sunou <fj1826dm(a)aa.jp.fujitsu.com> Atsushi SAKAI <sakaia(a)jp.fujitsu.com> Thanks, Masayuki Sunou.

3 4

[Libvir] Proposal: Check availability of driver calls (repost)
by Richard W.M. Jones 02 Aug '07

02 Aug '07

-- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

3 9

[Libvir] Exporting calls just to libvirtd
by Richard W.M. Jones 01 Aug '07

01 Aug '07

With the two patches[1][2] I recently posted, the remote driver has started to make calls more directly into the drivers. Calls like driver->domainMigratePrepare in [2] are exposed to the remote driver, but shouldn't be exported for public use. However because libvirtd links to regular libvirt.so, the calls actually do need to be exported, so we export them as __virDomainMigratePrepare (etc) with the double underscore and exclusion from <libvirt.h> indicating that users of the published libvirt API should not call them. It seems we could avoid exporting them entirely with one of these methods: (a) Link libvirtd to the static libvirt.a. -- Probably violates a load of distro policy and removes some supposed benefits of dynamic linking. (b) Build a separate shared library with a different dynamic symbol table. -- Also removes one benefit of dynamic linking, because the two libraries are different despite having the same code in them. Any ideas if some advanced feature of ELF offers a way out here? Rich. [1]https://www.redhat.com/archives/libvir-list/2007-July/msg00444.html [2]https://www.redhat.com/archives/libvir-list/2007-July/msg00357.html -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903

1 0