8:32 a.m.
A libvirt client can connect to libvirt on a remote host. However, on
its own this doesn't give the client a usable level of access. The
reason for this is that libvirt doesn't expose some critical management
interfaces. For QEMU this amounts to pretty much anything provided
directly by the QEMU process, including:
* VNC console [1]
* Any character device [2]
I'm undecided as to whether access to underlying storage falls into this
bucket. It would certainly be convenient in certain circumstances.
This means, for example that I need out of band access to:
* See a graphical console
* Connect to a serial console
* Connect to a guest channel
As these are fairly fundamental operations, I wonder if there has been
any thought towards creating tunnels over a libvirt connection.
In the first instance, I think this would require a generic method to
multiplex multiple streams over a single remote connection. This would
obviously be a significant protocol change. Would a messaging protocol,
AMQP for instance, be a good candidate here?
Secondly, I think interfaces which currently create local resources
should have a default which is managed by libvirt itself. So you'd have:
<serial>
<target port='0'/>
</serial>
or
<serial type='managed'>
<target port='0'/>
</serial>
This would cause libvirt to create whichever host-side chardev is most
convenient to it and connect itself. The API would then expose read and
write APIs for the named connection. e.g:
virDomainFoo foo;[3]
if = virDomainGetFoo(domain, "serial0");
bytesIn = virDomainFooRead(foo, &buf, sizeof(buf));
A client application could expose this in a variety of ways locally.
Matt
[1] There's an open RFE for this one:
https://bugzilla.redhat.com/show_bug.cgi?id=526953
[2] e.g.
virsh # console RHEL3.FV.64.IDE
error: Cannot connect to a remote console device
[3] What would you call it?
--
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team
M: +44 (0)7977 267231
GPG ID: D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
11:03 a.m.
On 12/11/2009 03:32 PM, Matthew Booth wrote:
A libvirt client can connect to libvirt on a remote host. However,
on
its own this doesn't give the client a usable level of access. The
reason for this is that libvirt doesn't expose some critical management
interfaces. For QEMU this amounts to pretty much anything provided
directly by the QEMU process, including:
* VNC console [1]
* Any character device [2]
I'm undecided as to whether access to underlying storage falls into this
bucket. It would certainly be convenient in certain circumstances.
This means, for example that I need out of band access to:
* See a graphical console
* Connect to a serial console
* Connect to a guest channel
As these are fairly fundamental operations, I wonder if there has been
any thought towards creating tunnels over a libvirt connection.
In the first instance, I think this would require a generic method to
multiplex multiple streams over a single remote connection. This would
obviously be a significant protocol change. Would a messaging protocol,
AMQP for instance, be a good candidate here?
In point of fact, we *do* have a way to mulitplex generic data through a
libvirt tunnel; it's how we implemented "tunnelled" migration for
kvm. Look in git for commits referencing "data streams", around August 2009.
I think danpb's plan was to eventually tunnel character devices
(serial console's in particular) over the generic data stream, but
I think that work got preempted by more high priority work.
So I think what you are proposing makes sense, it just needs someone
to sit down and plumb it. If you are gung-ho to do it, the implementation
for tunnelled migration is in src/qemu/qemu_driver.c to use as an example.
--
Chris Lalancette
3:48 p.m.
On Fri, Dec 11, 2009 at 02:32:36PM +0000, Matthew Booth wrote:
A libvirt client can connect to libvirt on a remote host. However, on
its own this doesn't give the client a usable level of access. The
reason for this is that libvirt doesn't expose some critical management
interfaces. For QEMU this amounts to pretty much anything provided
directly by the QEMU process, including:
* VNC console [1]
* Any character device [2]
I'm undecided as to whether access to underlying storage falls into this
bucket. It would certainly be convenient in certain circumstances.
This means, for example that I need out of band access to:
* See a graphical console
* Connect to a serial console
* Connect to a guest channel
As these are fairly fundamental operations, I wonder if there has been
any thought towards creating tunnels over a libvirt connection.
The long term vision that I've had for remote access is that we should
aim for 2 network services. The first providing host management. The
second providing guest interaction.
Currently libvirt RPC (or any of the bindings mentioned below) provide
the former, while VNC provides the latter. VNC has already gained the
ability to tunnel audio streams, and we've done a proof of concept to
make it tunnel serial/guest consoles over the same channel too. There
are a handful of other services that could be tunnelled over VNC too,
in particular it would be compelling to tunnel a remote CDROM over
it as you typically find with bladecenter mgmt consoles.
I think it is important to keep the guest interaction vs host management
network services separate for a security POV. The libvirtd daemon already
arguably has too many privileges with all the APIs it has to support, but
at least all those are targetted at the system administrator. Adding the
guest admin access into libvirtd makes the consquences of any flaw in it
far higher, because the guest admin is a totally untrusted user. This
would give them an avenue to raise their privileges.
From a network architecture POV too, the two services will likely be
delivered separately. For a cloud deployment, the guest services would
be exposed to the internet, but host mgmt would certainly be local to
the management LAN only. The libvirtd daemon is specifically single
host focused. With guest migration, the guest services should really
be thought of as host independant. We have had thought about writing
a VNC server proxy, to which end users. This proxy would run off node
and connection to arbitrary VNC servers in QEMU. Thus end users would
never be connecting to a single node, as they do with libvirt.
In the first instance, I think this would require a generic method to
multiplex multiple streams over a single remote connection. This would
obviously be a significant protocol change. Would a messaging protocol,
AMQP for instance, be a good candidate here?
As Chris notes, the libvirt RPC protocol already allows for tunnelling
different things over it, not even requiring them to be API calls.
The question though is whether we really want libvirt to become a
general purpose RPC service, when there plenty of others out there
already ? Taking AMQP as an example, there is already the libvirt-qpid.git
agent which exposes libvirt's API over AMQP. There is libvirt CIM which
exposes libvirt over CIM. Someone could create a libvirt XMLRPC agent
which exposed the APIs over HTTP. I don't think libvirt's RPC needs
any changes, since it has already demonstrated the flexibility to
integrate as needed.
Secondly, I think interfaces which currently create local resources
should have a default which is managed by libvirt itself. So you'd have:
<serial>
<target port='0'/>
</serial>
or
<serial type='managed'>
<target port='0'/>
</serial>
This would cause libvirt to create whichever host-side chardev is most
convenient to it and connect itself. The API would then expose read and
write APIs for the named connection. e.g:
With the way we are exposing audio (and in the future serial/guest console)
to VNC in QEMU there is not neccesarily any need for a particular config.
The VNC audio extension implementation in QEMU captures all data from the
guest sound cards, no matter how it is configured to output to the host.
Likewise our proof of ceontp for serial/guest console tunnelling over
VNC captured all data, regardless of where the chardev config was pointing
[2] e.g.
virsh # console RHEL3.FV.64.IDE
error: Cannot connect to a remote console device
This is the one thing where I would make an exception to everything I
have just said. Serial console access is an important recovery tool
for the virtualization host administrator. This implies that we should
make it available via the libvirt API. So I think we should try to add
a data stream for the serial console over libvirt, if only to make
virsh console work. For guest administrators though, I still think they
should be accessing the serial console over VNC.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5495
days inactive
5495
days old
2 comments
3 participants
participants (3)
-
Chris Lalancette -
Daniel P. Berrange -
Matthew Booth