5:35 a.m.
I'm playing with VMWare ESX 4.0 and I decided to try driving it from
libvirt [git version]. Here are some random questions and issues I
encountered, in no particular order.
Listing domains works fine, but:
(a) I have to specify ?no_verify=1 to get it to ignore the lack of
valid https certificate. Is there some documentation on how to set up
certificates correctly? [vSphere gives exactly the same error]
(b) I have to type the root password (of the ESX server) each time.
Can I avoid that?
$ sudo ./tools/virsh -c 'esx://192.168.2.121/?no_verify=1' list --all
Enter username for 192.168.2.121 [root]:
Enter root password for 192.168.2.121:
Id Name State
----------------------------------
- TestLinux shut off
- TestWin shut off
(c) Starting domains doesn't seem to work, and the error message is
obscure:
$ sudo ./tools/virsh -c 'esx://192.168.2.121/?no_verify=1' start TestLinux
Enter username for 192.168.2.121 [root]:
Enter root password for 192.168.2.121:
error: Failed to start domain TestLinux
error: internal error HTTP response code 500 for call to 'PowerOnVM_Task'. Fault:
ServerFaultCode - The operation is not allowed in the current state.
Is this to do with the "ask questions" API that was discussed on this
list before? Specifying auto_answer=0 or auto_answer=1 didn't make
any difference.
(d) dumpxml XML output looks very good, and consistent with the other
drivers. However it produces an odd <source file> attribute:
<disk type='file' device='disk'>
<driver name='lsilogic'/>
<source file='[Storage1] TestLinux/TestLinux.vmdk'/>
<target dev='sda' bus='scsi'/>
</disk>
<disk type='file' device='cdrom'>
<source file='[Storage1] Fedora-12-x86_64-Live.iso'/>
<target dev='hdc' bus='ide'/>
</disk>
(e) pool-list, net-list are not supported by the driver.
That's a particular problem for me because I was really hoping to use
the storage APIs from libvirt to access the VMDK files that contain
domains (eg. [Storage1] TestLinux/TestLinux.vmdk in the example
above). Is support for storage (particularly) and/or networks on the
horizon for this driver?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top
2:29 p.m.
2009/12/30 Richard W.M. Jones <rjones(a)redhat.com>:
I'm playing with VMWare ESX 4.0 and I decided to try driving it
from
libvirt [git version]. Here are some random questions and issues I
encountered, in no particular order.
Listing domains works fine, but:
(a) I have to specify ?no_verify=1 to get it to ignore the lack of
valid https certificate. Is there some documentation on how to set up
certificates correctly? [vSphere gives exactly the same error]
The default ESX certificates are autogenerated and self-signed ones.
The 'ESX Configuration Guide' [1] suggest to replace them by CA-signed
ones.
The 'ESX Configuration Guide' [1] contains a section about 'Replace a
Default Certificate with a CA-Signed Certificate'.
I created new self-signed certificates as described on the libvirt
website [2] and copied them to the ESX server
servercert.pem -> /etc/vmware/ssl/rui.crt
serverkey.pem -> /etc/vmware/ssl/rui.key
Then restart the hostd process either with
service mgmt-vmware restart
or if that doesn't work (like with my ESXi 3.5 on USB flash drive) use
/etc/init.d/hostd restart
On my Ubuntu box I copied the cacert.pem to
/usr/share/ca-certificates/esx-certs/cacert.pem, appended
esx-certs/cacert.pem to /etc/ca-certificates.conf and executed
update-ca-certificates. After this libcurl can verify the ESX server
certificate.
libvirt uses explicit paths for its certificates for a remote TLS
connection using gnutls. This is possible with libcurl too, using the
CURLOPT_CAINFO option. For example the ESX driver could use
LIBVIRT_CACERT as the remote driver does, but this would require the
ESX server certificates to be signed by the same CA as the libvirtd
certificates, because libcurl doesn't fall back to the system CA
certificates if verification with the explicitly specified CA fails.
(b) I have to type the root password (of the ESX server) each time.
Can I avoid that?
$ sudo ./tools/virsh -c 'esx://192.168.2.121/?no_verify=1' list --all
Enter username for 192.168.2.121 [root]:
Enter root password for 192.168.2.121:
Id Name State
----------------------------------
- TestLinux shut off
- TestWin shut off
The ESX driver needs to login to the ESX server. virsh uses the
default auth callback that asks you to type in the credentials (see
virConnectAuthCallbackDefault in libvirt.c). Currently the only option
for automatic handling is policykit, but that doesn't work for the ESX
driver because policykit is not meant to provide credentials.
A possible way to "automatically" provide the credentials is to pass
them in the connection URI like this: esx://<user>:<pass>@hostname.
But this is currently not implemented in the ESX driver.
Another option would be to add new automatic authentication methods to
virConnectAuthCallbackDefault based on VIR_CRED_EXTERNAL and to change
the ESX driver to use them.
(c) Starting domains doesn't seem to work, and the error message
is
obscure:
$ sudo ./tools/virsh -c 'esx://192.168.2.121/?no_verify=1' start TestLinux
Enter username for 192.168.2.121 [root]:
Enter root password for 192.168.2.121:
error: Failed to start domain TestLinux
error: internal error HTTP response code 500 for call to 'PowerOnVM_Task'. Fault:
ServerFaultCode - The operation is not allowed in the current state.
Is this to do with the "ask questions" API that was discussed on this
list before? Specifying auto_answer=0 or auto_answer=1 didn't make
any difference.
No, the error message would say that there is a question blocking the
task. This message comes directly from the ESX server, as part of the
500er response.
I've never seen this error message from the ESX server before. Many
errors reported by the ESX server contain a details section, but the
deserialization of this details requires many new SOAP types and I
haven't implemented this yet.
Can you start the domain using the VI client? Maybe the domain really
is in a state where it cannot be started.
Can you reproduce this error with other domains?
(d) dumpxml XML output looks very good, and consistent with the
other
drivers. However it produces an odd <source file> attribute:
<disk type='file' device='disk'>
<driver name='lsilogic'/>
<source file='[Storage1] TestLinux/TestLinux.vmdk'/>
<target dev='sda' bus='scsi'/>
</disk>
<disk type='file' device='cdrom'>
<source file='[Storage1] Fedora-12-x86_64-Live.iso'/>
<target dev='hdc' bus='ide'/>
</disk>
What's wrong with it in your opinion? The <source file> attribute is
correct. See the ESX driver website section [3] about this.
(e) pool-list, net-list are not supported by the driver.
That's a particular problem for me because I was really hoping to use
the storage APIs from libvirt to access the VMDK files that contain
domains (eg. [Storage1] TestLinux/TestLinux.vmdk in the example
above). Is support for storage (particularly) and/or networks on the
horizon for this driver?
Rich.
I plan to implement the ESX driver as complete as possible, this
includes network and storage handling. I've took a look at this some
time ago and saw some problems. For examples datastores don't have a
UUID, so I may need to make them up and store them somewhere like the
IBM Power driver does it for the domain UUIDs, the libvirt network
model cannot represent a vSwitch ... I think these drivers are much
harder to implemented than the main ESX driver, so don't expect any
news on this in the next weeks, but stay tuned :)
What do you mean by "to access the VMDK files that contain domains",
just list them?
What part of the storage driver would that involve?
[1] http://www.vmware.com/pdf/vsphere4/r40/vsp_40_esx_server_config.pdf
[2] http://www.libvirt.org/remote.html#Remote_certificates
[3] http://www.libvirt.org/drvesx.html#xmlspecial
Matthias
3:51 a.m.
On Wed, Dec 30, 2009 at 09:29:10PM +0100, Matthias Bolte wrote:
I've never seen this error message from the ESX server before.
Many
errors reported by the ESX server contain a details section, but the
deserialization of this details requires many new SOAP types and I
haven't implemented this yet.
Can we get LIBVIRT_DEBUG=1 to just dump the SOAP response?
/me checks the source ...
It seems like currently debug output is hard-coded into a #define.
Having it wired to LIBVIRT_DEBUG would allow people to debug these
issues without needing to recompile.
Can you start the domain using the VI client? Maybe the domain
really
is in a state where it cannot be started.
OK, turns out that the VMWare server was in "maintenance mode".
What's wrong with it in your opinion? The <source file>
attribute is
correct. See the ESX driver website section [3] about this.
Nothing wrong with it, just a bit unusual. But as you say, it is a
valid path according to ESX's unusual path convention.
> (e) pool-list, net-list are not supported by the driver.
[...]
I plan to implement the ESX driver as complete as possible, this
includes network and storage handling. I've took a look at this some
time ago and saw some problems. For examples datastores don't have a
UUID, so I may need to make them up and store them somewhere like the
IBM Power driver does it for the domain UUIDs, the libvirt network
model cannot represent a vSwitch ... I think these drivers are much
harder to implemented than the main ESX driver, so don't expect any
news on this in the next weeks, but stay tuned :)
What do you mean by "to access the VMDK files that contain domains",
just list them?
It's so that we can inspect the VM's disk, for virt-inspector:
http://libguestfs.org/virt-inspector.1.html
Ideally we'd like to download all or part of the VM's disk (ie. VMDK
file(s)).
Thanks for your detailed reply!
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
6:42 p.m.
New subject: [libvirt] QEMU/KVM snapshots, and rudimentary API extension
I was starting to use libvirt on a project, and realized that there was nothing in the API
for snapshots. As I need to be able to take snapshots without taking down the machine
(that and KVM's restore function didn't seem to be working terribly well), I added
a few functions so that I could take snapshots, and I also added a screenshot function so
I could give users a view of their snapshots in my application. At this point, as I only
really use KVM/QEMU, that is the only hypervisor I implemented it for, and I'm sure
there will be some problems with my structure in the other drivers. However, I figured I
should at least submit the patch, even if it turns out to be of no use to anyone :)
At this point for the QEMU driver, it only does snapshots for qcow2 drives, and will
report an error if qemu says that no valid drives were found, or if for restoring/deleting
the specified snapshot wasn't found.
Philip Jameson
2:49 p.m.
New subject: [libvirt] QEMU/KVM snapshots, and rudimentary API extension
On 12/31/2009 07:42 PM, Philip Jameson wrote:
I was starting to use libvirt on a project, and realized that there
was nothing in the API for snapshots. As I need to be able to take snapshots without
taking down the machine (that and KVM's restore function didn't seem to be working
terribly well), I added a few functions so that I could take snapshots, and I also added a
screenshot function so I could give users a view of their snapshots in my application. At
this point, as I only really use KVM/QEMU, that is the only hypervisor I implemented it
for, and I'm sure there will be some problems with my structure in the other drivers.
However, I figured I should at least submit the patch, even if it turns out to be of no
use to anyone :)
At this point for the QEMU driver, it only does snapshots for qcow2 drives, and will
report an error if qemu says that no valid drives were found, or if for restoring/deleting
the specified snapshot wasn't found.
Hi Philip,
Thanks for the patch, snapshot handling is definitely one of the big
missing pieces in libvirt. For this patch to get better attention, I'd
recommend making a top level post (for some reason this mail appeared as
a reply to an unrelated thread), and split the patch into discreet
parts. You can see an example of such a split at
http://libvirt.org/api_extension.html. Using git send-email will make
this process very easy.
It's probably also a good idea to split the screenshot API from the
snapshot pieces and submit them separately.
FYI, Dan (the primary libvirt architect) did a write up a while ago
describing what he thought a snapshotting API should look like:
http://www.redhat.com/archives/libvir-list/2009-March/msg00205.html
It's pretty similar to yours: it might help to adjust your API entry
points to take into account Dan's ideas.
Thanks for the contribution!
- Cole
5:54 a.m.
2009/12/31 Richard W.M. Jones <rjones(a)redhat.com>:
On Wed, Dec 30, 2009 at 09:29:10PM +0100, Matthias Bolte wrote:
> I've never seen this error message from the ESX server before. Many
> errors reported by the ESX server contain a details section, but the
> deserialization of this details requires many new SOAP types and I
> haven't implemented this yet.
Can we get LIBVIRT_DEBUG=1 to just dump the SOAP response?
/me checks the source ...
It seems like currently debug output is hard-coded into a #define.
Having it wired to LIBVIRT_DEBUG would allow people to debug these
issues without needing to recompile.
I assume you refer to ESX_VI__CURL__ENABLE_DEBUG_OUTPUT and
esxVI_CURL_Debug. I just noticed that this is broken and also doesn't
dump the transfered SOAP messages.
I don't think that dumping the complete SOAP communication using
VIR_DEBUG is useful in general, but I'm going to dump the
undeserialized SOAP faults so the message details are available in the
debug output.
> Can you start the domain using the VI client? Maybe the domain
really
> is in a state where it cannot be started.
OK, turns out that the VMWare server was in "maintenance mode".
Ah. I should add a check to the ESX driver and output a warning or at
least an info about the ESX server being in maintenance mode.
> What's wrong with it in your opinion? The <source file>
attribute is
> correct. See the ESX driver website section [3] about this.
Nothing wrong with it, just a bit unusual. But as you say, it is a
valid path according to ESX's unusual path convention.
> > (e) pool-list, net-list are not supported by the driver.
[...]
> I plan to implement the ESX driver as complete as possible, this
> includes network and storage handling. I've took a look at this some
> time ago and saw some problems. For examples datastores don't have a
> UUID, so I may need to make them up and store them somewhere like the
> IBM Power driver does it for the domain UUIDs, the libvirt network
> model cannot represent a vSwitch ... I think these drivers are much
> harder to implemented than the main ESX driver, so don't expect any
> news on this in the next weeks, but stay tuned :)
>
> What do you mean by "to access the VMDK files that contain domains",
> just list them?
It's so that we can inspect the VM's disk, for virt-inspector:
http://libguestfs.org/virt-inspector.1.html
Ideally we'd like to download all or part of the VM's disk (ie. VMDK
file(s)).
Hm, I haven't looked into the storage driver API in detail yet, but
maybe it allows report an URL to access a volume. An ESX server
exposes its datastores (local and remote) and all files in them over
https://<hostname>/folder. You can also put files there using this
mechanism.
Thanks for your detailed reply!
Rich.
Matthias
5431
days inactive
5440
days old
5 comments
4 participants
participants (4)
-
Cole Robinson -
Matthias Bolte -
Philip Jameson -
Richard W.M. Jones