[libvirt] libvirt-qpid
Howdy folks! So I've been working on a qpid interface to libvirt. Here's the description I've come up with: libvirt-qpid provides an interface with libvirt using QMF (qpid modeling framework) which utilizes the AMQP protocol. The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol providing reliable transport of messages. QMF provides a modeling framework layer on top of qpid. This interface allows you to manage hosts, domains, pools etc. as a set of objects with properties and methods. The source code repo: http://git.et.redhat.com/?p=libvirt-qpid.git;a=summary So I'd like to hear any feedback you guys might have. This file in particular describes the API that is used to interact with libvirt. http://git.et.redhat.com/?p=libvirt-qpid.git;a=blob_plain;f=src/libvirt-sche... My TODO list looks like this: - Not sure on camel case or underscores. Original libvirt API uses camel case, I went with underscores as I was originally basing it off the ruby bindings. - Need rc scripts, daemonization, better reconnect/error handling for both qpid and libvirt. I think it should attempt to reconnect until successful whenever it gets disconnected. We may need an extra status property for this state as well. - Return libvirt error codes? (it's just text msgs right now with error status bit). - Make it less chatty, but log stuff If you wish to try it out, I've made rpms for fedora 9 x86_64 and i386. They are in the ovirt repo, which you can add to yum using: rpm -ivh http://ovirt.org/repos/ovirt/9/ovirt-release-LATEST.fc9.noarch.rpm Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals): qpidd --auth no libvirt-qpid (as root to auth with libvirt) qpid-tool qpid-tool provides an interface to qpid and allows you to view/manipulate the qpid models/objects. Here is an example of how it looks using qpid-tool: $ qpid-tool Management Tool for QPID qpid: list Management Object Types: ObjectType Active Deleted ============================================ com.redhat.libvirt:domain 2 0 com.redhat.libvirt:node 1 0 qpid: list node Objects of type com.redhat.libvirt:node ID Created Destroyed Index ================================= 101 00:02:00 - qpid: show 101 Object of type com.redhat.libvirt:node: (last sample time: 17:47:54) Type Element 101 ============================================== property hostname vpro.mains.net property uri qemu:///system property libvirt_version 0.4.5 property api_version 0.4.5 property hypervisor_version 0.9.1 property hypervisor_type QEMU qpid: list domain Objects of type com.redhat.libvirt:domain ID Created Destroyed Index ================================= 102 00:02:05 - 101 103 00:02:05 - 101 qpid: show 102 Object of type com.redhat.libvirt:domain: (last sample time: 17:48:09) Type Element 102 ================================================================= property uuid 662958f8-12c5-baad-2252-4d2126e0971f property name ovirt-appliance property node 101 property state shutoff property num_vcpus 1 property active false statistic maximum_memory 786432 statistic memory 786432 statistic cpu_time 0 qpid: call 102 create Call Result: create 0 (OK) {} qpid: show 102 Object of type com.redhat.libvirt:domain: (last sample time: 17:48:24) Type Element 102 ================================================================= property uuid 662958f8-12c5-baad-2252-4d2126e0971f property name ovirt-appliance property node 101 property state running property num_vcpus 1 property active true statistic maximum_memory 786432 statistic memory 786432 statistic cpu_time 0 qpid:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
So I've been working on a qpid interface to libvirt. Here's the description I've come up with:
libvirt-qpid provides an interface with libvirt using QMF (qpid modeling framework) which utilizes the AMQP protocol. The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol providing reliable transport of messages.
QMF provides a modeling framework layer on top of qpid. This interface allows you to manage hosts, domains, pools etc. as a set of objects with properties and methods.
The source code repo:
http://git.et.redhat.com/?p=libvirt-qpid.git;a=summary
So I'd like to hear any feedback you guys might have. This file in particular describes the API that is used to interact with libvirt.
http://git.et.redhat.com/?p=libvirt-qpid.git;a=blob_plain;f=src/libvirt-sche...
I really like how you decided to go for a 1-to-1 mapping of libvirt C API into QMF api :-) I am still kinda undecided about the Node vs Connect root object, but I'm leaning towards your current model, or 1 libvirt-qpid agent using a specific libvirt connect URI, thus making an explicit Connect object unneccessary. If someone wants to export multiple hypervisor connections that it ought to be possible to run multiple instances of the daemon Also, if we want to have a real accurate 1-to-1 mapping, then really the Domain statistics would be better in a DomainInfo object, in the same way you've dealt with Node, vs NodeInfo objects.
My TODO list looks like this:
- Not sure on camel case or underscores. Original libvirt API uses camel case, I went with underscores as I was originally basing it off the ruby bindings.
Typically this is decided according to conventions of the language being bound to. Are there any established conventions by people doing QMF work ? If there are none, then I'd recommend we stick to the convention used by the libvirt C library.
- Need rc scripts, daemonization, better reconnect/error handling for both qpid and libvirt. I think it should attempt to reconnect until successful whenever it gets disconnected. We may need an extra status property for this state as well.
You can probably just copy the libvirtd init script & rename the daemon process in it - particularly if you follow the libvirtd argv parsing. Auto-reconnecting upon failure is a interesting idea - be good to be able to restart libvirtd, without needing to bounce the agent too
- Return libvirt error codes? (it's just text msgs right now with error status bit).
Yep, we need to include as much of the libvirt error object data as possible - merely skip the 'domain' and 'network' deprecated fields, and the 'connect' field which is irrelevant iun this context.
- Make it less chatty, but log stuff
Configurable use of syslog() would be a good idea. For sake of consistency I'd recommend looking at the libvirtd daemon, and its command line arg handling - see qemud/qemud.c file and the main() method in it. Pretty much all the libvirtd argv would make sense for libvirt-qpid agent.
If you wish to try it out, I've made rpms for fedora 9 x86_64 and i386. They are in the ovirt repo, which you can add to yum using:
rpm -ivh http://ovirt.org/repos/ovirt/9/ovirt-release-LATEST.fc9.noarch.rpm
Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals):
qpidd --auth no libvirt-qpid (as root to auth with libvirt) qpid-tool
Have you tried it out across with a network of machines all routing over a AMQP grid ? I'm interested to know how 'chatty' it is as we add lots of libvirt nodes - particularly wrt to the pushing of domain statistics on a frequent basis for lots of domains.
qpid-tool provides an interface to qpid and allows you to view/manipulate the qpid models/objects. Here is an example of how it looks using qpid-tool:
This is perhaps the bit I like most about QPID - automatically get a single point global view of the system, with a reliable messaging backbone, and strong authentication & strong typing. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, Oct 01, 2008 at 11:10:41AM +0100, Daniel P. Berrange wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
- Make it less chatty, but log stuff
Configurable use of syslog() would be a good idea. For sake of consistency I'd recommend looking at the libvirtd daemon, and its command line arg handling - see qemud/qemud.c file and the main() method in it. Pretty much all the libvirtd argv would make sense for libvirt-qpid agent.
Likewise, you'll probably want a configuration file for stuff like the IP address rather than passing everything on the command line. A file in /etc/libvirt/libvirt-qpid.conf would be a good location, preferrably following the same syntax & config arg naming as the existing libvirtd.conf there. It'd be fairly easy to take our 'virConf' stuff in src/conf.c and use it in your daemon - just c++-ify it as desired. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, 1 Oct 2008 11:36:24 +0100 "Daniel P. Berrange" <berrange@redhat.com> wrote:
On Wed, Oct 01, 2008 at 11:10:41AM +0100, Daniel P. Berrange wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
- Make it less chatty, but log stuff
Configurable use of syslog() would be a good idea. For sake of consistency I'd recommend looking at the libvirtd daemon, and its command line arg handling - see qemud/qemud.c file and the main() method in it. Pretty much all the libvirtd argv would make sense for libvirt-qpid agent.
Likewise, you'll probably want a configuration file for stuff like the IP address rather than passing everything on the command line. A file in /etc/libvirt/libvirt-qpid.conf would be a good location, preferrably following the same syntax & config arg naming as the existing libvirtd.conf there. It'd be fairly easy to take our 'virConf' stuff in src/conf.c and use it in your daemon - just c++-ify it as desired.
Well I'm glad you generally approve :). I've done a number of things since this announcement: - added getopt_long option parsing. - to solve the config issue I used /etc/sysconfig/libvirt-qpid environment variable to pass in broker host and port. Maybe a config file would be better I'm not sure. - daemonization/init script. So it basically works on an ovirt node now. I'm just getting some bugs fixed and it should "work". Still some things to do like reconnect, libvirt error codes etc. but it's coming. I'll let you know how chatty it gets when there's lots of nodes. :) Ian
On Wed, Oct 01, 2008 at 02:55:57PM -0700, Ian Main wrote:
On Wed, 1 Oct 2008 11:36:24 +0100 "Daniel P. Berrange" <berrange@redhat.com> wrote:
On Wed, Oct 01, 2008 at 11:10:41AM +0100, Daniel P. Berrange wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
- Make it less chatty, but log stuff
Configurable use of syslog() would be a good idea. For sake of consistency I'd recommend looking at the libvirtd daemon, and its command line arg handling - see qemud/qemud.c file and the main() method in it. Pretty much all the libvirtd argv would make sense for libvirt-qpid agent.
Likewise, you'll probably want a configuration file for stuff like the IP address rather than passing everything on the command line. A file in /etc/libvirt/libvirt-qpid.conf would be a good location, preferrably following the same syntax & config arg naming as the existing libvirtd.conf there. It'd be fairly easy to take our 'virConf' stuff in src/conf.c and use it in your daemon - just c++-ify it as desired.
Well I'm glad you generally approve :).
I've done a number of things since this announcement:
- added getopt_long option parsing. - to solve the config issue I used /etc/sysconfig/libvirt-qpid environment variable to pass in broker host and port. Maybe a config file would be better I'm not sure.
A combination of both is usually desired. The recommended split of responsibility is as follows: - /etc/sysconfig/libvirt-qpid - control how the daemon is started and controlled by the boot process. This is usually restricted to things like passing a --daemon flag (if neeed), configuring syslog levels, but not stuff with has a functional effect on the daemon. So not IP addresses / ports / etc. It also usually has a catch all generic variable to allow the admin to give arbitrary command line args. - /etc/libvirt/libvirt-qpid.conf - control functional behaviour of the daemon. TCP port/addresses, authentication (if applicable), connection URI, important stuff like that. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, 1 Oct 2008 11:10:41 +0100 "Daniel P. Berrange" <berrange@redhat.com> wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
So I've been working on a qpid interface to libvirt. Here's the description I've come up with:
libvirt-qpid provides an interface with libvirt using QMF (qpid modeling framework) which utilizes the AMQP protocol. The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol providing reliable transport of messages.
QMF provides a modeling framework layer on top of qpid. This interface allows you to manage hosts, domains, pools etc. as a set of objects with properties and methods.
The source code repo:
http://git.et.redhat.com/?p=libvirt-qpid.git;a=summary
So I'd like to hear any feedback you guys might have. This file in particular describes the API that is used to interact with libvirt.
http://git.et.redhat.com/?p=libvirt-qpid.git;a=blob_plain;f=src/libvirt-sche...
I really like how you decided to go for a 1-to-1 mapping of libvirt C API into QMF api :-)
I am still kinda undecided about the Node vs Connect root object, but I'm leaning towards your current model, or 1 libvirt-qpid agent using a specific libvirt connect URI, thus making an explicit Connect object unneccessary. If someone wants to export multiple hypervisor connections that it ought to be possible to run multiple instances of the daemon
Also, if we want to have a real accurate 1-to-1 mapping, then really the Domain statistics would be better in a DomainInfo object, in the same way you've dealt with Node, vs NodeInfo objects.
Hmm.. I guess I just wonder why you would separate them. From a 'look at the objects' point of view they may as well just be in the class. But then I'm guessing that would be more for compatibility. Actually, another way to look at it is that it is really just a bunch of info about a domain that is returned in a structure rather than a single call like many of the other properties already in the domain. :) hehe, if it's a big deal I can change it :).
My TODO list looks like this:
- Not sure on camel case or underscores. Original libvirt API uses camel case, I went with underscores as I was originally basing it off the ruby bindings.
Typically this is decided according to conventions of the language being bound to. Are there any established conventions by people doing QMF work ? If there are none, then I'd recommend we stick to the convention used by the libvirt C library.
It sounds like we're going camel. qmf uses camelcase too. Sounds like for the rest we're in agreement. Just some time to implement is needed. :)
qpid-tool provides an interface to qpid and allows you to view/manipulate the qpid models/objects. Here is an example of how it looks using qpid-tool:
This is perhaps the bit I like most about QPID - automatically get a single point global view of the system, with a reliable messaging backbone, and strong authentication & strong typing.
Yeah it's pretty neat. Ian
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
Howdy folks!
Okay i started playing with this a bit, so a few hints and feedback
The source code repo:
http://git.et.redhat.com/?p=libvirt-qpid.git;a=summary
So I'd like to hear any feedback you guys might have. This file in particular describes the API that is used to interact with libvirt.
http://git.et.redhat.com/?p=libvirt-qpid.git;a=blob_plain;f=src/libvirt-sche...
My TODO list looks like this:
- Not sure on camel case or underscores. Original libvirt API uses camel case, I went with underscores as I was originally basing it off the ruby bindings.
I would keep with camel case to maximize similarity with the libvirt API ideally someone looking just at libvirt documentation or code should be able to make his way on the libvirt-qpid API just based on the similarities.
- Need rc scripts, daemonization, better reconnect/error handling for both qpid and libvirt. I think it should attempt to reconnect until successful whenever it gets disconnected. We may need an extra status property for this state as well.
with an limited exponential backoff timeout to not waste too many cycles on unconnected laptops :-)
- Return libvirt error codes? (it's just text msgs right now with error status bit).
Like Dan suggested try to keep the same kind of structure and fields, except for the couple of deprecated ones.
- Make it less chatty, but log stuff
If you wish to try it out, I've made rpms for fedora 9 x86_64 and i386. They are in the ovirt repo, which you can add to yum using:
rpm -ivh http://ovirt.org/repos/ovirt/9/ovirt-release-LATEST.fc9.noarch.rpm
Note to interested users, you also need to enable the repo by editing /etc/yum.repos.d/ovirt.repo after install and switch enable to 1 at least in the first section.
Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals):
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
qpidd --auth no libvirt-qpid (as root to auth with libvirt) qpid-tool
qpid-tool provides an interface to qpid and allows you to view/manipulate the qpid models/objects. Here is an example of how it looks using qpid-tool:
$ qpid-tool Management Tool for QPID qpid: list
Well ... all I got there was Waiting for next periodic update on the other hand the libvirt-qpid is checking every second or so the current state, I guess the connection isn't made or something. I tried qpid-tool both as root and $user BTW i would suggest to rename qpid-tool to libvirt-qpid-tool or virt-qpid-tool to avoid confusion about the scope. Some comments about the XML schemas: - camelCaseDanke :-) - Node has methods domain_define_xml, storage_pool_define_xml and storage_pool_create_xml I think at least for symetry domain_create_xml should be available there too. - I would make an Error class mimicking at least partially what's available in libvirt C or python bindings - Domain.create() is very confusing, again I would define domain creation under node, i.e. temporary 'undefined' domains. Then <method name="create" desc="Start stopped VM"/> should probably be renamed 'start' to avoid confusion. and the comment "Start a defined but stopped domain" would be more adequate by mentioning the define API... - Is there a way to map constants in schemas, I think they should be exposed at the API level, for example migration flags should have an explicit definition here ... if possible. That sounds cool, I don't know why it doesn't work for me, I still see no indocation of error in the 3 consoles but still failing for me after 20mn... qpid: list Waiting for next periodic update qpid: Any verbose option to try to find why this is not carried on the bus or if the bus is just not established ? Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
Howdy folks!
If you wish to try it out, I've made rpms for fedora 9 x86_64 and i386. They are in the ovirt repo, which you can add to yum using:
rpm -ivh http://ovirt.org/repos/ovirt/9/ovirt-release-LATEST.fc9.noarch.rpm
Note to interested users, you also need to enable the repo by editing /etc/yum.repos.d/ovirt.repo after install and switch enable to 1 at least in the first section.
Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals):
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
qpidd --auth no libvirt-qpid (as root to auth with libvirt) qpid-tool
qpid-tool provides an interface to qpid and allows you to view/manipulate the qpid models/objects. Here is an example of how it looks using qpid-tool:
$ qpid-tool Management Tool for QPID qpid: list
Well ... all I got there was
Waiting for next periodic update
on the other hand the libvirt-qpid is checking every second or so the current state, I guess the connection isn't made or something. I tried qpid-tool both as root and $user
BTW i would suggest to rename qpid-tool to libvirt-qpid-tool or virt-qpid-tool to avoid confusion about the scope.
Actually qpid-tool is a general purpose shell provided by qpid themselves, not a libvirt specific thing - if you have other agents active it'll show those objects as wel as the libvirt ones.
Some comments about the XML schemas: - camelCaseDanke :-) - Node has methods domain_define_xml, storage_pool_define_xml and storage_pool_create_xml I think at least for symetry domain_create_xml should be available there too.
Yep, all APIs should be expressed in the qpid binding eventually.
- I would make an Error class mimicking at least partially what's available in libvirt C or python bindings - Domain.create() is very confusing, again I would define domain creation under node, i.e. temporary 'undefined' domains. Then <method name="create" desc="Start stopped VM"/> should probably be renamed 'start' to avoid confusion. and the comment "Start a defined but stopped domain" would be more adequate by mentioning the define API...
The trouble with that is it would diverge from libvirt naming - admittedly the libvirt naming isn't ideal, but i figure consistency is better so people can cross-reference API documentation more easily. So the 'create' method on Domain does make sense, but we'd expect another 'createLinux' method on the Node object for unmanaged domains - though it might be worth breaking with consistency in this one case and dropping the 'Linux' suffix from the name there . Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Thu, Oct 02, 2008 at 03:40:06PM +0100, Daniel P. Berrange wrote:
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
Ah, that probably explains why that didn't work for me, I got my qpidd from Fedora ... Another question, upon rebout the QPid service starts automatically, I wonder if there is something to do to be able to connect assuming a "default install" and not starting without auth .
BTW i would suggest to rename qpid-tool to libvirt-qpid-tool or virt-qpid-tool to avoid confusion about the scope.
Actually qpid-tool is a general purpose shell provided by qpid themselves, not a libvirt specific thing - if you have other agents active it'll show those objects as wel as the libvirt ones.
Oops my bad, i didn't realize it came from qpid itself, good to know.
Some comments about the XML schemas: - camelCaseDanke :-) - Node has methods domain_define_xml, storage_pool_define_xml and storage_pool_create_xml I think at least for symetry domain_create_xml should be available there too.
Yep, all APIs should be expressed in the qpid binding eventually.
- I would make an Error class mimicking at least partially what's available in libvirt C or python bindings - Domain.create() is very confusing, again I would define domain creation under node, i.e. temporary 'undefined' domains. Then <method name="create" desc="Start stopped VM"/> should probably be renamed 'start' to avoid confusion. and the comment "Start a defined but stopped domain" would be more adequate by mentioning the define API...
The trouble with that is it would diverge from libvirt naming - admittedly the libvirt naming isn't ideal, but i figure consistency is better so people can cross-reference API documentation more easily. So the 'create' method on Domain does make sense,
okay but let's improve the comment then, it's really confusing
but we'd expect another 'createLinux' method on the Node object for unmanaged domains - though it might be worth breaking with consistency in this one case and dropping the 'Linux' suffix from the name there .
Yeah, the Linux suffix is a remain from the very early days with paravirt. Actually we should fix libvirt itself and just provide the old symbol for compatibility. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Thu, 2 Oct 2008 16:50:08 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Thu, Oct 02, 2008 at 03:40:06PM +0100, Daniel P. Berrange wrote:
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
Ah, that probably explains why that didn't work for me, I got my qpidd from Fedora ... Another question, upon rebout the QPid service starts automatically, I wonder if there is something to do to be able to connect assuming a "default install" and not starting without auth .
Ah, I know what happened.. I changed the dependencies since my first email. qpidd isn't needed for libvirt-qpid to operate. It's just an agent and can talk to a qpidd on a different host. Definitely want to make sure you get all the latest versions from the ovirt repo. Fedora has previous release of qpid.. ones in ovirt repo are all devel snapshots. qpid has codefreeze coming in 3 weeks or so for next release.
BTW i would suggest to rename qpid-tool to libvirt-qpid-tool or virt-qpid-tool to avoid confusion about the scope.
Actually qpid-tool is a general purpose shell provided by qpid themselves, not a libvirt specific thing - if you have other agents active it'll show those objects as wel as the libvirt ones.
Oops my bad, i didn't realize it came from qpid itself, good to know.
Some comments about the XML schemas: - camelCaseDanke :-) - Node has methods domain_define_xml, storage_pool_define_xml and storage_pool_create_xml I think at least for symetry domain_create_xml should be available there too.
Yep, all APIs should be expressed in the qpid binding eventually.
- I would make an Error class mimicking at least partially what's available in libvirt C or python bindings - Domain.create() is very confusing, again I would define domain creation under node, i.e. temporary 'undefined' domains. Then <method name="create" desc="Start stopped VM"/> should probably be renamed 'start' to avoid confusion. and the comment "Start a defined but stopped domain" would be more adequate by mentioning the define API...
The trouble with that is it would diverge from libvirt naming - admittedly the libvirt naming isn't ideal, but i figure consistency is better so people can cross-reference API documentation more easily. So the 'create' method on Domain does make sense,
okay but let's improve the comment then, it's really confusing
but we'd expect another 'createLinux' method on the Node object for unmanaged domains - though it might be worth breaking with consistency in this one case and dropping the 'Linux' suffix from the name there .
Yeah, the Linux suffix is a remain from the very early days with paravirt. Actually we should fix libvirt itself and just provide the old symbol for compatibility.
Good idea :) I was wondering why that wasn't the case.. Ian
On Thu, Oct 02, 2008 at 08:16:02AM -0700, Ian Main wrote:
On Thu, 2 Oct 2008 16:50:08 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Thu, Oct 02, 2008 at 03:40:06PM +0100, Daniel P. Berrange wrote:
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
Ah, that probably explains why that didn't work for me, I got my qpidd from Fedora ... Another question, upon rebout the QPid service starts automatically, I wonder if there is something to do to be able to connect assuming a "default install" and not starting without auth .
Ah, I know what happened.. I changed the dependencies since my first email. qpidd isn't needed for libvirt-qpid to operate. It's just an agent and can talk to a qpidd on a different host.
okay, I was guessing something like that which is why i was so cautious in my wording ;-)
Definitely want to make sure you get all the latest versions from the ovirt repo. Fedora has previous release of qpid.. ones in ovirt repo are all devel snapshots. qpid has codefreeze coming in 3 weeks or so for next release.
[root@wei ~]# rpm -q qpidd qpidd-0.3.698276-1.fc9.x86_64 [root@wei ~]# rpm -qi qpidd | grep Build Release : 1.fc9 Build Date: Tue 23 Sep 2008 09:13:31 PM CEST Install Date: Thu 02 Oct 2008 02:33:41 PM CEST Build Host: vpro.mains.net [root@wei ~]# I assume I got the updated version on update.
but we'd expect another 'createLinux' method on the Node object for unmanaged domains - though it might be worth breaking with consistency in this one case and dropping the 'Linux' suffix from the name there .
Yeah, the Linux suffix is a remain from the very early days with paravirt. Actually we should fix libvirt itself and just provide the old symbol for compatibility.
Good idea :) I was wondering why that wasn't the case..
Okay, I will submit a patch here. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Thu, Oct 02, 2008 at 05:33:54PM +0200, Daniel Veillard wrote:
On Thu, Oct 02, 2008 at 08:16:02AM -0700, Ian Main wrote:
On Thu, 2 Oct 2008 16:50:08 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Thu, Oct 02, 2008 at 03:40:06PM +0100, Daniel P. Berrange wrote:
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
Ah, that probably explains why that didn't work for me, I got my qpidd from Fedora ... Another question, upon rebout the QPid service starts automatically, I wonder if there is something to do to be able to connect assuming a "default install" and not starting without auth .
Ah, I know what happened.. I changed the dependencies since my first email. qpidd isn't needed for libvirt-qpid to operate. It's just an agent and can talk to a qpidd on a different host.
okay, I was guessing something like that which is why i was so cautious in my wording ;-)
Okay I now have things working ... on one machine. Which is interesting but not really the point of the exercise :-) I have 3 machines. qpidd runs on machine A, qpid-tool on that machine allows to access the local node and local domains. libvirt-qpid started on machine B and C in the same subnet without error, but well they don't find the qpidd on machine A since I don't see them there. qpid-tool there tries to get to localhost and fail. If then I install and start qpidd on machine B qpid-tool can connect to it ... but the already started libvirt-qpid don't seems to be able to find it, unless I restart it which seems to indicate a failure to connect after startup. And I'm still only able to list local domains/nodes never remote ones. Ideas ? Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Fri, Oct 03, 2008 at 03:29:17PM +0200, Daniel Veillard wrote:
On Thu, Oct 02, 2008 at 05:33:54PM +0200, Daniel Veillard wrote:
On Thu, Oct 02, 2008 at 08:16:02AM -0700, Ian Main wrote:
On Thu, 2 Oct 2008 16:50:08 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Thu, Oct 02, 2008 at 03:40:06PM +0100, Daniel P. Berrange wrote:
On Thu, Oct 02, 2008 at 03:06:01PM +0200, Daniel Veillard wrote:
You also need 'yum install qpidd' I suspect this indicates a missing dependancy maybe in the libvirt-qpid package but I'm not 100% sure
Yeah, i believe that should be a requirement. NB, the versions of qpidc and qpidd in Fedora currently are too old for libvirt-qpid. I've been speaking with qpid maintainer and they'll push a new build to Fedora in the very near future.
Ah, that probably explains why that didn't work for me, I got my qpidd from Fedora ... Another question, upon rebout the QPid service starts automatically, I wonder if there is something to do to be able to connect assuming a "default install" and not starting without auth .
Ah, I know what happened.. I changed the dependencies since my first email. qpidd isn't needed for libvirt-qpid to operate. It's just an agent and can talk to a qpidd on a different host.
okay, I was guessing something like that which is why i was so cautious in my wording ;-)
Okay I now have things working ... on one machine. Which is interesting but not really the point of the exercise :-) I have 3 machines. qpidd runs on machine A, qpid-tool on that machine allows to access the local node and local domains. libvirt-qpid started on machine B and C in the same subnet without error, but well they don't find the qpidd on machine A since I don't see them there. qpid-tool there tries to get to localhost and fail. If then I install and start qpidd on machine B qpid-tool can connect to it ... but the already started libvirt-qpid don't seems to be able to find it, unless I restart it which seems to indicate a failure to connect after startup. And I'm still only able to list local domains/nodes never remote ones.
Ideas ?
I got this working across 3 machines as follows - Machine A provides a Qpid broker, run as root with qpidd --auth no - Machine B and C are libvirt hosts, each run a libvirtd, and libvirt-qpid libvirt-qpid --broker machineA.example.com On machine A, if you run 'qpid-tool' you should now see node, domain objects from both machines B & C. NB, it take 5-10 seconds from starting libvirt-qpid before they appear in the broker Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Fri, Oct 03, 2008 at 02:50:08PM +0100, Daniel P. Berrange wrote:
If then I install and start qpidd on machine B qpid-tool can connect to it ... but the already started libvirt-qpid don't seems to be able to find it, unless I restart it which seems to indicate a failure to connect after startup. And I'm still only able to list local domains/nodes never remote ones.
Ideas ?
I got this working across 3 machines as follows
- Machine A provides a Qpid broker, run as root with
qpidd --auth no
- Machine B and C are libvirt hosts, each run a libvirtd, and libvirt-qpid
libvirt-qpid --broker machineA.example.com
On machine A, if you run 'qpid-tool' you should now see node, domain objects from both machines B & C. NB, it take 5-10 seconds from starting libvirt-qpid before they appear in the broker
I got this working it's just that my firewall rules on node A blocked access ! Basically one need to open TCP port 5672 on the machine running the QPid broker (and possibly 5671 when SSL auth is to be used once configured). Also the simplest is to edit /etc/qpidd.conf on the broker node and add : auth=no then restart the broker with "/etc/init.d/qpidd restart" and on all the monitored nodes edit /etc/sysconfig/libvirt-qpid and add : LIBVIRT_QPID_ARGS="--broker 192.168.0.11" and also restart the libvirt-qpid service on them. Assuming the services gets started automatically all the nodes will connect automatically to the broker upon reboot without manual tuning. very cool ! I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Mon, Oct 06, 2008 at 04:18:15PM +0200, Daniel Veillard wrote:
I got this working it's just that my firewall rules on node A blocked access ! Basically one need to open TCP port 5672 on the machine running the QPid broker (and possibly 5671 when SSL auth is to be used once configured). Also the simplest is to edit /etc/qpidd.conf on the broker node and add :
auth=no
then restart the broker with "/etc/init.d/qpidd restart" and on all the monitored nodes edit /etc/sysconfig/libvirt-qpid and add :
LIBVIRT_QPID_ARGS="--broker 192.168.0.11"
and also restart the libvirt-qpid service on them. Assuming the services gets started automatically all the nodes will connect automatically to the broker upon reboot without manual tuning.
very cool !
I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node.
That is something oVirt does make use of, but I don't think it is useful in the general case deployment of the daemon, because regardless of whether you can find the broker, the admin needs to configure each machine with authentication credentials. So I'd just use of SRV records upto apps like oVirt to manage. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Mon, 6 Oct 2008 16:18:15 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Fri, Oct 03, 2008 at 02:50:08PM +0100, Daniel P. Berrange wrote:
If then I install and start qpidd on machine B qpid-tool can connect to it ... but the already started libvirt-qpid don't seems to be able to find it, unless I restart it which seems to indicate a failure to connect after startup. And I'm still only able to list local domains/nodes never remote ones.
Ideas ?
I got this working across 3 machines as follows
- Machine A provides a Qpid broker, run as root with
qpidd --auth no
- Machine B and C are libvirt hosts, each run a libvirtd, and libvirt-qpid
libvirt-qpid --broker machineA.example.com
On machine A, if you run 'qpid-tool' you should now see node, domain objects from both machines B & C. NB, it take 5-10 seconds from starting libvirt-qpid before they appear in the broker
I got this working it's just that my firewall rules on node A blocked access ! Basically one need to open TCP port 5672 on the machine running the QPid broker (and possibly 5671 when SSL auth is to be used once configured). Also the simplest is to edit /etc/qpidd.conf on the broker node and add :
auth=no
then restart the broker with "/etc/init.d/qpidd restart" and on all the monitored nodes edit /etc/sysconfig/libvirt-qpid and add :
LIBVIRT_QPID_ARGS="--broker 192.168.0.11"
and also restart the libvirt-qpid service on them. Assuming the services gets started automatically all the nodes will connect automatically to the broker upon reboot without manual tuning.
very cool !
I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node.
Nice, glad you got it all working! :) This is exactly how we do it in ovirt.. I posted a patch to use DNS SRV entries and it just sets up the libvirt-qpid sysconfig file on each boot. Ian
On Mon, Oct 06, 2008 at 08:14:12AM -0700, Ian Main wrote:
I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node.
Nice, glad you got it all working! :)
This is exactly how we do it in ovirt.. I posted a patch to use DNS SRV entries and it just sets up the libvirt-qpid sysconfig file on each boot.
Trying to make sure I understand, there is no such support in the upstream qpid but that's a patch carried just on Ovirt ? Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Mon, Oct 06, 2008 at 05:54:28PM +0200, Daniel Veillard wrote:
On Mon, Oct 06, 2008 at 08:14:12AM -0700, Ian Main wrote:
I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node.
Nice, glad you got it all working! :)
This is exactly how we do it in ovirt.. I posted a patch to use DNS SRV entries and it just sets up the libvirt-qpid sysconfig file on each boot.
Trying to make sure I understand, there is no such support in the upstream qpid but that's a patch carried just on Ovirt ?
It is not really a patch - it is just the way oVirt configures its managed node virtualization host at startup. The managed node has a whole bunch of services running, of which libvirtqpid is just one. So, after networking comes up, oVirt queries the SRV records for all the services it cares about, and uses augeas to edit the config files to point to these services. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Mon, Oct 06, 2008 at 04:57:45PM +0100, Daniel P. Berrange wrote:
On Mon, Oct 06, 2008 at 05:54:28PM +0200, Daniel Veillard wrote:
On Mon, Oct 06, 2008 at 08:14:12AM -0700, Ian Main wrote:
I'm just wondering if there is any DNS customization trick which would allow the qpid client to automatically find the broker based on the DNS settings (things like _qpid._tcp SRV entries in the local domain). That would avoid the configuration on each monitored node.
Nice, glad you got it all working! :)
This is exactly how we do it in ovirt.. I posted a patch to use DNS SRV entries and it just sets up the libvirt-qpid sysconfig file on each boot.
Trying to make sure I understand, there is no such support in the upstream qpid but that's a patch carried just on Ovirt ?
It is not really a patch - it is just the way oVirt configures its managed node virtualization host at startup.
The managed node has a whole bunch of services running, of which libvirtqpid is just one. So, after networking comes up, oVirt queries the SRV records for all the services it cares about, and uses augeas to edit the config files to point to these services.
Ah, I was hoping some kind of standardization of lookup embedded in the qpid library itself. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals):
qpidd --auth no libvirt-qpid (as root to auth with libvirt)
I'm a little bit worried by the following: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 9758 qpidd 20 0 273m 137m 1644 S 46.6 6.8 986:30.29 qpidd that's with just 3 nodes over a week with no traffic except the keeping the machines on the bus, the memory footprint of qpidd seems to be growing, very slowly but nonetheless continuously, and that's just with 3 nodes hooked with libvirt-qpid. is that a genuine leak in qpidd ? (qpidd-0.3.700546-1.fc9.x86_64) is that related to the policy to keep messages sent to the QPid bus ? I wonder. It also consumes some non-neglective amount of CPU as a background task. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
On Mon, 13 Oct 2008 15:56:21 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote:
Once you have that set up, 'yum install libvirt-qpid python-qpid', and then run (each in their own terminals):
qpidd --auth no libvirt-qpid (as root to auth with libvirt)
I'm a little bit worried by the following:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 9758 qpidd 20 0 273m 137m 1644 S 46.6 6.8 986:30.29 qpidd
that's with just 3 nodes over a week with no traffic except the keeping the machines on the bus, the memory footprint of qpidd seems to be growing, very slowly but nonetheless continuously, and that's just with 3 nodes hooked with libvirt-qpid. is that a genuine leak in qpidd ? (qpidd-0.3.700546-1.fc9.x86_64) is that related to the policy to keep messages sent to the QPid bus ?
I wonder. It also consumes some non-neglective amount of CPU as a background task.
I talked to the qpid folks about this today. It's a bug one way or another. They fixed one that may be causing this and there will be new RPMs out hopefully today. If that doesn't do it it may have something to do with how QMF is doing things. If you still see this after the RPM upgrade let me know. I'll be trying it out too. Thanks! Ian
On Thu, Oct 16, 2008 at 12:34:05PM -0700, Ian Main wrote:
On Mon, 13 Oct 2008 15:56:21 +0200 Daniel Veillard <veillard@redhat.com> wrote:
On Wed, Sep 24, 2008 at 10:55:36AM -0700, Ian Main wrote: I'm a little bit worried by the following:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 9758 qpidd 20 0 273m 137m 1644 S 46.6 6.8 986:30.29 qpidd
that's with just 3 nodes over a week with no traffic except the keeping the machines on the bus, the memory footprint of qpidd seems to be growing, very slowly but nonetheless continuously, and that's just with 3 nodes hooked with libvirt-qpid. is that a genuine leak in qpidd ? (qpidd-0.3.700546-1.fc9.x86_64) is that related to the policy to keep messages sent to the QPid bus ?
I wonder. It also consumes some non-neglective amount of CPU as a background task.
I talked to the qpid folks about this today. It's a bug one way or another. They fixed one that may be causing this and there will be new RPMs out hopefully today. If that doesn't do it it may have something to do with how QMF is doing things. If you still see this after the RPM upgrade let me know. I'll be trying it out too. Thanks!
Okay, I will keep an eye, thanks for the info ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
participants (3)
-
Daniel P. Berrange -
Daniel Veillard -
Ian Main