[libvirt] the role of libvirtd
Hello all. I spent quite a while yesterday in a meeting with a client interested in using libvirt for a large project. They are quite happy in general with the direction things are going. However they have objections to a couple of libvirt design points that have come up in multiple cases in the past, including in developing the other project I'm involved in, oVirt. Specifically, they would really prefer that libvirtd be a one-stop shop for everything they need to do on a virtualization host, including things we have traditionally held out-of-scope for libvirt. A partial list of those things would include: * In-depth multipath config management * Hardware lifecycle management (power-off, reboot, etc.) * HA configuration ... and pretty much anything else you might want to do on a piece of hardware that is hosting virtual machines. My initial reaction of course was to tell them "Well you should just use a separate agent for that sort of thing." But of course this means making a separate connection to the node depending on what operation you want to perform, which is cumbersome. So I guess what I'm asking is, why *not* expand the scope of libvirtd to be a one-stop shop for managing a node? Is there a really good reason it shouldn't have the remaining capabilities libvirt users want? Is that reason good enough to balance the headache our users have to deal with in coming up with a separate package to handle the tasks libvirtd does not? Slings, arrows, or thunderous applause welcome... Take care, --Hugh
On Wed, 2009-07-08 at 12:10 -0400, Hugh O. Brock wrote:
My initial reaction of course was to tell them "Well you should just use a separate agent for that sort of thing." But of course this means making a separate connection to the node depending on what operation you want to perform, which is cumbersome.
Allow tunnelling arbitrary TCP connections over the libvirt connection? i.e. SSH tunnels, but replacing the SSH bit with libvirt :-) Cheers, Mark.
On Wed, Jul 08, 2009 at 12:10:18PM -0400, Hugh O. Brock wrote:
Hello all.
I spent quite a while yesterday in a meeting with a client interested in using libvirt for a large project.
They are quite happy in general with the direction things are going. However they have objections to a couple of libvirt design points that have come up in multiple cases in the past, including in developing the other project I'm involved in, oVirt. Specifically, they would really prefer that libvirtd be a one-stop shop for everything they need to do on a virtualization host, including things we have traditionally held out-of-scope for libvirt. A partial list of those things would include:
* In-depth multipath config management * Hardware lifecycle management (power-off, reboot, etc.) * HA configuration
... and pretty much anything else you might want to do on a piece of hardware that is hosting virtual machines.
My initial reaction of course was to tell them "Well you should just use a separate agent for that sort of thing." But of course this means making a separate connection to the node depending on what operation you want to perform, which is cumbersome.
So I guess what I'm asking is, why *not* expand the scope of libvirtd to be a one-stop shop for managing a node? Is there a really good reason it shouldn't have the remaining capabilities libvirt users want? Is that reason good enough to balance the headache our users have to deal with in coming up with a separate package to handle the tasks libvirtd does not?
This is essentially suggesting that libvirtd become a general purpose RPC layer for all remote management tasks. At which point you have just re-invented QPid/AMQP or CIM or any number of other general purpose message buses. libvirtd has a core well defined goal: - Provide a remote proxy for libvirt API calls if you want todo anything more than that you should be considering an alternative remote management system. We already have 2 good ones to choose from supported with libvirt - QPid/AMQP, with libvirt-qpid agent + your own custom agents - CIM, with libvirt-CIM + your own custom CIM providers Both of these offer other benefits besides just pluggable support for other functionality. In particular - Non-blocking asynchronous RPC calls - Assured delivery for RPC calls - Scalable network architecture / topology - Inter-operability with plugins written by other projects/vendors Furthermore, adding more plugins to libvirtd means we will never be able to reduce its privileges to an acceptable level, because we'll never know what capabilities the plugins may want. Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, Jul 08, 2009 at 05:46:43PM +0100, Daniel P. Berrange wrote:
On Wed, Jul 08, 2009 at 12:10:18PM -0400, Hugh O. Brock wrote:
Hello all.
I spent quite a while yesterday in a meeting with a client interested in using libvirt for a large project.
They are quite happy in general with the direction things are going. However they have objections to a couple of libvirt design points that have come up in multiple cases in the past, including in developing the other project I'm involved in, oVirt. Specifically, they would really prefer that libvirtd be a one-stop shop for everything they need to do on a virtualization host, including things we have traditionally held out-of-scope for libvirt. A partial list of those things would include:
* In-depth multipath config management * Hardware lifecycle management (power-off, reboot, etc.) * HA configuration
... and pretty much anything else you might want to do on a piece of hardware that is hosting virtual machines.
My initial reaction of course was to tell them "Well you should just use a separate agent for that sort of thing." But of course this means making a separate connection to the node depending on what operation you want to perform, which is cumbersome.
So I guess what I'm asking is, why *not* expand the scope of libvirtd to be a one-stop shop for managing a node? Is there a really good reason it shouldn't have the remaining capabilities libvirt users want? Is that reason good enough to balance the headache our users have to deal with in coming up with a separate package to handle the tasks libvirtd does not?
This is essentially suggesting that libvirtd become a general purpose RPC layer for all remote management tasks. At which point you have just re-invented QPid/AMQP or CIM or any number of other general purpose message buses.
libvirtd has a core well defined goal:
- Provide a remote proxy for libvirt API calls
if you want todo anything more than that you should be considering an alternative remote management system. We already have 2 good ones to choose from supported with libvirt
- QPid/AMQP, with libvirt-qpid agent + your own custom agents - CIM, with libvirt-CIM + your own custom CIM providers
Both of these offer other benefits besides just pluggable support for other functionality. In particular
- Non-blocking asynchronous RPC calls - Assured delivery for RPC calls - Scalable network architecture / topology - Inter-operability with plugins written by other projects/vendors
Furthermore, adding more plugins to libvirtd means we will never be able to reduce its privileges to an acceptable level, because we'll never know what capabilities the plugins may want.
I understand your point -- certainly we want to use existing RPC mechanisms for libvirt and node management, not maintain our own. However, given a libvirt-qpid daemon on the node that handles RPC over QMF (for example), is there not some value in having libvirt expose a consistent API for the operations people want to do on a host regardless of whether they have directly to do with managing a virtual machine or not? I will note that when I presented the large client with the option of QMF talking to multiple agents on the node but exposing (effectively) a single API and a single connection, they seemed much happier. So perhaps the right way to attack this is with the ovirt-qpid daemon we are currently working on. Daniel V., any further thoughts on this? --Hugh
On Wed, Jul 08, 2009 at 01:39:44PM -0400, Hugh O. Brock wrote:
On Wed, Jul 08, 2009 at 05:46:43PM +0100, Daniel P. Berrange wrote:
This is essentially suggesting that libvirtd become a general purpose RPC layer for all remote management tasks. At which point you have just re-invented QPid/AMQP or CIM or any number of other general purpose message buses.
libvirtd has a core well defined goal:
- Provide a remote proxy for libvirt API calls
if you want todo anything more than that you should be considering an alternative remote management system. We already have 2 good ones to choose from supported with libvirt
- QPid/AMQP, with libvirt-qpid agent + your own custom agents - CIM, with libvirt-CIM + your own custom CIM providers
Both of these offer other benefits besides just pluggable support for other functionality. In particular
- Non-blocking asynchronous RPC calls - Assured delivery for RPC calls - Scalable network architecture / topology - Inter-operability with plugins written by other projects/vendors
Furthermore, adding more plugins to libvirtd means we will never be able to reduce its privileges to an acceptable level, because we'll never know what capabilities the plugins may want.
I understand your point -- certainly we want to use existing RPC mechanisms for libvirt and node management, not maintain our own.
However, given a libvirt-qpid daemon on the node that handles RPC over QMF (for example), is there not some value in having libvirt expose a consistent API for the operations people want to do on a host regardless of whether they have directly to do with managing a virtual machine or not?
I don't really see any value in that - you're just putting in another abstraction layer where none need exist. Just have whatever QMF agent you write talk directly to the thing you need to manage. If someone wants to write a QMF agent to managing cluster software, they don't need to introduce an artificial dependancy on libvirtd, when their agent could talk directly to the cluster software being managed, and thus be useful without libvirt deployed.
I will note that when I presented the large client with the option of QMF talking to multiple agents on the node but exposing (effectively) a single API and a single connection, they seemed much happier. So perhaps the right way to attack this is with the ovirt-qpid daemon we are currently working on.
A client application cannot tell whether a remote service is implemented by a single agent, or multiple agents, nor do they see the concept of a connection. All they see is a set of objects, representing everything on the message bus. So again for clients, there is no need for everything to be in one agent. Regards, Daniel -- |: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
On Wed, Jul 08, 2009 at 05:46:43PM +0100, Daniel P. Berrange wrote:
On Wed, Jul 08, 2009 at 12:10:18PM -0400, Hugh O. Brock wrote:
Hello all.
I spent quite a while yesterday in a meeting with a client interested in using libvirt for a large project.
They are quite happy in general with the direction things are going. However they have objections to a couple of libvirt design points that have come up in multiple cases in the past, including in developing the other project I'm involved in, oVirt. Specifically, they would really prefer that libvirtd be a one-stop shop for everything they need to do on a virtualization host, including things we have traditionally held out-of-scope for libvirt. A partial list of those things would include:
* In-depth multipath config management * Hardware lifecycle management (power-off, reboot, etc.) * HA configuration
... and pretty much anything else you might want to do on a piece of hardware that is hosting virtual machines.
My initial reaction of course was to tell them "Well you should just use a separate agent for that sort of thing." But of course this means making a separate connection to the node depending on what operation you want to perform, which is cumbersome.
So I guess what I'm asking is, why *not* expand the scope of libvirtd to be a one-stop shop for managing a node? Is there a really good reason it shouldn't have the remaining capabilities libvirt users want? Is that reason good enough to balance the headache our users have to deal with in coming up with a separate package to handle the tasks libvirtd does not?
This is essentially suggesting that libvirtd become a general purpose RPC layer for all remote management tasks. At which point you have just re-invented QPid/AMQP or CIM or any number of other general purpose message buses.
libvirtd has a core well defined goal:
- Provide a remote proxy for libvirt API calls
I'm a bit synpathetic to the suggestion though. I think libvirt API should help run those virtualization nodes, I would not open the gate like completely, but if we could provide all APIs needed to manage the node on a day by day basis then I think this is not really beyond our scope. I think that netcf is an example of such API where we start to add admin services for the purpose of running virtualization. Things like rebooting or shutting down the node would fit in this, maybe editing a drive partition too. HA configuration starts to be a bit stretched, I would expect this to be set once at creation and not part of the routine maintainance, so probably out of scope, multipath is a bit more in scope we discussed this already. Basically if we take the idea of a stripped down Node used only for virtualization, then except for operations which are first time setup options or maintainance, I think we should try to cover the requirements of normal operations of that node. To some extend that means we would step on the toes of CIM, but we would stick to a subset that's sure. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Hugh O. Brock -
Mark McLoughlin