On Fri, Apr 11, 2014 at 1:23 PM, Daniel P. Berrange <berrange@redhat.com> wrote:

On Fri, Apr 11, 2014 at 01:14:50PM +0200, Ruben Kerkhof wrote:

...

Hi all,

I have a few python scripts which use the libvirt api to get interface and block device statistics. What has been bugging me for a while now that is that there’s no high level api to get a list of all interfaces or block devices for a vm. The list can be retrieved from the xml with a bit of Xpath magic, but this seems to me to break the nice abstraction layer libvirt provides. Ideally, I don’t have to do anything with xml, and add dependencies on xml parsers to my code.

I’ve seen examples of code doing this, for example the collectd libvirt plugin, but there must be many others.

Can I kindly ask for such an API? Unfortunately I don’t have the skills to code this up myself.

The libvirt-glib project provides a set of libraries including libvirt-gconfig which intend to fill the gap you describe. They are accessible from Python using GObject Introspection support. The goal of libvirt-gconfig is to remove the need for apps to know anything about XML at all.

So now instead of having to learn XML parsing I'll have to learn GObject and glib? And instead of a dependency on an xml parser I'll have to add one on glib. Not exactly the easy solution I was hoping for ;-) I still think that a method to get the interfaces for a vm is a gap in the current api, which is working fine for all my other needs. I can get the interfaces for a node, so why not for a vm?

eg,this demo program shows how to build a domain config using the APIs. For parsing you just pass it the XML document and then can use the various getters to access info

#!/usr/bin/python

from gi.repository import LibvirtGConfig;

domain = LibvirtGConfig.Domain.new() domain.set_virt_type(LibvirtGConfig.DomainVirtType.KVM) domain.set_name("foo") domain.set_memory(1024*1024) # 1 GB domain.set_vcpus(2) domain.set_lifecycle(LibvirtGConfig.DomainLifecycleEvent.ON_POWEROFF, LibvirtGConfig.DomainLifecycleAction.DESTROY) domain.set_virt_type(LibvirtGConfig.DomainVirtType.KVM)

clock = LibvirtGConfig.DomainClock.new() clock.set_offset(LibvirtGConfig.DomainClockOffset.UTC) domain.set_clock(clock)

os = LibvirtGConfig.DomainOs.new() os.set_os_type(LibvirtGConfig.DomainOsType.HVM) os.set_arch("x86_64") devices = [ LibvirtGConfig.DomainOsBootDevice.CDROM, LibvirtGConfig.DomainOsBootDevice.NETWORK ] os.set_boot_devices(devices) domain.set_os(os)

disk = LibvirtGConfig.DomainDisk.new() disk.set_type(LibvirtGConfig.DomainDiskType.FILE) disk.set_guest_device_type(LibvirtGConfig.DomainDiskGuestDeviceType.DISK) disk.set_source("/tmp/foo/bar") disk.set_driver_name("qemu") disk.set_driver_format(LibvirtGConfig.DomainDiskFormat.QCOW2) disk.set_target_bus(LibvirtGConfig.DomainDiskBus.IDE) disk.set_target_dev("hda") domain.add_device(disk)

interface = LibvirtGConfig.DomainInterfaceNetwork.new() interface.set_source("default") filterref = LibvirtGConfig.DomainInterfaceFilterref.new() filterref.set_name("clean-traffic") parameter = LibvirtGConfig.DomainInterfaceFilterrefParameter.new() parameter.set_name("IP") parameter.set_value("205.23.12.40") filterref.add_parameter(parameter) interface.set_filterref(filterref) domain.add_device(interface)

interface = LibvirtGConfig.DomainInterfaceUser.new() interface.set_ifname("eth0") interface.set_link_state(LibvirtGConfig.DomainInterfaceLinkState.UP) interface.set_mac("00:11:22:33:44:55") interface.set_model("foo") domain.add_device(interface)

input = LibvirtGConfig.DomainInput.new() input.set_device_type(LibvirtGConfig.DomainInputDeviceType.TABLET) input.set_bus(LibvirtGConfig.DomainInputBus.USB) domain.add_device(input)

graphics = LibvirtGConfig.DomainGraphicsSpice.new() graphics.set_port(1234) domain.add_device(graphics)

video = LibvirtGConfig.DomainVideo.new() video.set_model(LibvirtGConfig.DomainVideoModel.QXL) domain.add_device(video)

console = LibvirtGConfig.DomainConsole.new() pty = LibvirtGConfig.DomainChardevSourcePty.new() console.set_source(pty) domain.add_device(console)

print domain.to_xml()

pool = LibvirtGConfig.StoragePool.new() pool.set_pool_type(LibvirtGConfig.StoragePoolType.DIR)

pool_source = LibvirtGConfig.StoragePoolSource.new() pool_source.set_directory("/foo/bar") pool.set_source(pool_source)

perms = LibvirtGConfig.StoragePermissions.new() perms.set_owner(1001) perms.set_group(1005) perms.set_mode(0744) perms.set_label("virt_image_t")

pool_target = LibvirtGConfig.StoragePoolTarget.new() pool_target.set_path("/dev/disk/by-path") pool_target.set_permissions(perms) pool.set_target(pool_target)

print pool.to_xml()

vol = LibvirtGConfig.StorageVol.new() vol.set_name("my-vol") vol.set_capacity(0xdeadbeef)

vol_target = LibvirtGConfig.StorageVolTarget.new() vol_target.set_format("qcow2") vol_target.set_permissions(perms) vol.set_target(vol_target)

print vol.to_xml()

Thanks for the example, I'll dig into it. I just read https://www.berrange.com/posts/2011/11/22/introducing-the-libvirt-glib-a-map... Is there any documentation available for libvirt-gobject online? Unfortunately I'm still stuck on Scientific Linux 6.5 There are no packages for libvirt-glib in the Scientific Linux repo, are they available for libvirt 0.10 on RHEL 6.5?

Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

Kind regards, Ruben

On Fri, Apr 11, 2014 at 2:11 PM, Daniel P. Berrange <berrange@redhat.com> wrote:

On Fri, Apr 11, 2014 at 01:56:31PM +0200, Ruben Kerkhof wrote:

...

On Fri, Apr 11, 2014 at 1:23 PM, Daniel P. Berrange <berrange@redhat.com> wrote:

...

On Fri, Apr 11, 2014 at 01:14:50PM +0200, Ruben Kerkhof wrote:

...

Hi all,

I have a few python scripts which use the libvirt api to get interface and block device statistics. What has been bugging me for a while now that is that there’s no high level api to get a list of all interfaces or block devices for a vm. The list can be retrieved from the xml with a bit of Xpath magic, but this seems to me to break the nice abstraction layer libvirt provides. Ideally, I don’t have to do anything with xml, and add dependencies on xml parsers to my code.

I’ve seen examples of code doing this, for example the collectd libvirt plugin, but there must be many others.

Can I kindly ask for such an API? Unfortunately I don’t have the skills to code this up myself.

The libvirt-glib project provides a set of libraries including libvirt-gconfig which intend to fill the gap you describe. They are accessible from Python using GObject Introspection support. The goal of libvirt-gconfig is to remove the need for apps to know anything about XML at all.

So now instead of having to learn XML parsing I'll have to learn GObject and glib? And instead of a dependency on an xml parser I'll have to add one on glib. Not exactly the easy solution I was hoping for ;-)

Since you're using this from python, you don't have to learn / use any of glib / gobject, beyond the way you right the "import" statements. As the example I gave showed, the libvirt gconfig API just appears mapped into the regular Python object model. A dep on glib is really totally inconsequential compared to the deps that the main libvirt.so already has.

You're right, I just played a bit with it and it looks easy enough.

This effort wouldn't be practical without using GObject, because this frees us from having to write bindings for every programming language which has proved a major timesink with main libvirt library.

I understand.

...

Thanks for the example, I'll dig into it. I just read https://www.berrange.com/posts/2011/11/22/introducing-the-libvirt-glib-a-map... Is there any documentation available for libvirt-gobject online?

The -devel RPMs include the gtk-doc API docs, but I will freely admit they are pretty horribly incomplete. The easiest way to see APIs available is to use python's built-in introspection eg

...

...

...

from gi.repository import LibvirtGConfig; print dir(LibvirtGConfig.Domain) ['__class__', '__copy__', '__deepcopy__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__gdoc__', '__ge__', '__getattribute__', '__gpointer__', '__grefcount__', '__gsignals__', '__gt__', '__gtype__', '__hash__', '__info__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '_force_floating', '_ref', '_ref_sink', '_unref', '_unsupported_data_method', '_unsupported_method', 'add_device', 'bind_property', 'bind_property_full', 'chain', 'compat_control', 'connect', 'connect_after', 'connect_object', 'connect_object_after', 'disconnect', 'disconnect_by_func', 'emit', 'emit_stop_by_name', 'error_quark', 'force_floating', 'freeze_notify', 'g_type_instance', 'get_cpu', 'get_current_memory', 'get_custom_xml', 'get_data', 'get_description', 'get_devices', 'get_features', 'get_memory', 'get_name', 'get_os', 'get_properties', 'get_property', 'get_qdata', 'get_schema', 'get_title', 'get_vcpus', 'get_virt_type', 'handler_block', 'handler_block_by_func', 'handler_disconnect', 'handler_is_connected', 'handler_unblock', 'handler_unblock_by_func', 'interface_find_property', 'interface_install_property', 'interface_list_properties', 'is_floating', 'new', 'new_from_xml', 'notify', 'notify_by_pspec', 'parent', 'priv', 'props', 'qdata', 'ref', 'ref_count', 'ref_sink', 'replace_data', 'replace_qdata', 'run_dispose', 'set_clock', 'set_cpu', 'set_current_memory', 'set_custom_xml', 'set_data', 'set_description', 'set_devices', 'set_features', 'set_lifecycle', 'set_memory', 'set_name', 'set_os', 'set_power_management', 'set_properties', 'set_property', 'set_seclabel', 'set_title', 'set_vcpus', 'set_virt_type', 'steal_data', 'steal_qdata', 'stop_emission', 'stop_emission_by_name', 'thaw_notify', 'to_xml', 'unref', 'validate', 'watch_closure', 'weak_ref']

...

Unfortunately I'm still stuck on Scientific Linux 6.5 There are no packages for libvirt-glib in the Scientific Linux repo, are they available for libvirt 0.10 on RHEL 6.5?

Ah, I'm afraid you're out of luck on RHEL-6. This is only something we cna support on Fedora or forthcoming RHEL-7.

Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

Kind regards, Ruben

Hi Cole, On Fri, Apr 11, 2014 at 2:50 PM, Cole Robinson <crobinso@redhat.com> wrote:

On 04/11/2014 07:14 AM, Ruben Kerkhof wrote:

...

Hi all,

I have a few python scripts which use the libvirt api to get interface and block device statistics. What has been bugging me for a while now that is that there’s no high level api to get a list of all interfaces or block devices for a vm. The list can be retrieved from the xml with a bit of Xpath magic, but this seems to me to break the nice abstraction layer libvirt provides. Ideally, I don’t have to do anything with xml, and add dependencies on xml parsers to my code.

I’ve seen examples of code doing this, for example the collectd libvirt plugin, but there must be many others.

Can I kindly ask for such an API? Unfortunately I don’t have the skills to code this up myself.

It's an unavoidable fact that XML is part of the libvirt API. Going down the route of providing APIs that return bits and pieces of the XML is a slippery slope and increases libvirt maintenance burden.

python has a native XML library. To do what you want is pretty straight forward once you understand the concepts. For example this prints every interface mac address for the VM 'f20':

import xml.etree.ElementTree as ET import libvirt

conn = libvirt.open("qemu:///system") dom = conn.lookupByName("f20") xml = dom.XMLDesc(0)

root = ET.fromstring(xml) ifaces = root.findall("./devices/interface/mac") for iface in ifaces: print iface.attrib["address"]

You're right, it's not that hard. I've been using something like the following code for a while now: def get_interfaces(dom): interfaces = {} tree = ElementTree.fromstring(dom.XMLDesc(0)) for interface in tree.findall("devices/interface"): target = interface.find("target") if target is None: continue dev = target.get("dev") mac = interface.find("mac").get("address") if not dev in interfaces: interfaces[dev] = mac return interfaces Kind regards, Ruben