2:34 p.m.
I have been experimenting with using libvirt (0.3.3) on a variety of systems (RHEL, CentOS
and Oracle VM). I have run into an issue when I try to open a local read-only connection
to the hypervisor that is failing only on Oracle VM server release 2.2.0. I have created
a root owned setuid executable that is effectively running as root, but even so, still
cannot open the local read-only connection of the hypervisor. It only works if I run it
directly as root. This is not an option. I do not understand why it works as is on my
RHEL and CentOS machines, but not my Oracle machine. It would seem as thought it is not
checking if the effective uid is root, just the uid.
Has anyone run into a similar issue or have any suggestions of what I might try to fix
this issue or can tell me that this is a defect that needs (is) fixed??
Thanks.
6:57 a.m.
On Thu, Feb 25, 2010 at 02:34:02PM -0600, Tavares, John wrote:
I have been experimenting with using libvirt (0.3.3) on a variety of
systems (RHEL, CentOS and Oracle VM). I have run into an issue when
I try to open a local read-only connection to the hypervisor that is
failing only on Oracle VM server release 2.2.0. I have created a root
owned setuid executable that is effectively running as root, but even
so, still cannot open the local read-only connection of the hypervisor.
It only works if I run it directly as root. This is not an option. I
do not understand why it works as is on my RHEL and CentOS machines, but
not my Oracle machine. It would seem as thought it is not checking if
the effective uid is root, just the uid.
A readonly connection essentially just goes to a separate UNIX socket, which
has more relaxed permissions (mode 0777, instead of 0700). The kernel does
the permissions checking when attempting to open it, so it should be using
the effective ID.
Has anyone run into a similar issue or have any suggestions of what I
might try to fix this issue or can tell me that this is a defect that
needs (is) fixed??
I'd suggest trying to strace your process & see where it fails, and/or run
with LIBVIRT_DEBUG=1 environemnt variable set.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:42 a.m.
Thanks for your suggestions Daniel! I will rerun my executable thru strace with
LIBVIRT_DEBUG=1 to see what happens and will send the results if need be.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Monday, March 01, 2010 7:58 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Thu, Feb 25, 2010 at 02:34:02PM -0600, Tavares, John wrote:
I have been experimenting with using libvirt (0.3.3) on a variety of
systems (RHEL, CentOS and Oracle VM). I have run into an issue when
I try to open a local read-only connection to the hypervisor that is
failing only on Oracle VM server release 2.2.0. I have created a root
owned setuid executable that is effectively running as root, but even
so, still cannot open the local read-only connection of the hypervisor.
It only works if I run it directly as root. This is not an option. I
do not understand why it works as is on my RHEL and CentOS machines, but
not my Oracle machine. It would seem as thought it is not checking if
the effective uid is root, just the uid.
A readonly connection essentially just goes to a separate UNIX socket, which
has more relaxed permissions (mode 0777, instead of 0700). The kernel does
the permissions checking when attempting to open it, so it should be using
the effective ID.
Has anyone run into a similar issue or have any suggestions of what I
might try to fix this issue or can tell me that this is a defect that
needs (is) fixed??
I'd suggest trying to strace your process & see where it fails, and/or run
with LIBVIRT_DEBUG=1 environemnt variable set.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
6:38 a.m.
Hi Daniel,
Sorry it took so long to back to this. I just recently ran into the same problem on RHEL
5.5 where I am running into problems using the root owned setuid executable to get domain
information. Basically all that I needed to do to show this was to modify the example
program dominfo/info1.c that ships with the libvirt source and added a function that
attempts to get any information about that domain. In this case, I chose the
virDomainGetSchedulerType() function. Here is the output of 3 separate runs, the first
will be as a non-root user, the second is from root and the last is from the same
executable but just root owned setuid and run as a normal user:
$ ll info1
-r-xr-xr-x 1 perform perform 9326 Jul 12 16:50 info1
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
$ su
Password:
# ./info1
Domains 0: 2 CPUs
Scheduler = credit
# chown 0:0 info1;chmod 6555 info1
# ll info1
-r-sr-sr-x 1 root root 9326 Jul 12 16:50 info1
# exit
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
libvir: Xen Daemon error : this function is not supported by the hypervisor: unsupported
in xendConfigVersion < 4
$ virsh version
error: failed to connect to the hypervisor
$ su root -c "virsh version"
Password:
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
As you can see, even virsh needs to be run as root in order to connect to the hypervisor.
To help, I have attached both the source code and the executable I was using. Please let
me know if anything else is required to get this problem resolved.
Thanks.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Monday, March 01, 2010 7:58 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Thu, Feb 25, 2010 at 02:34:02PM -0600, Tavares, John wrote:
I have been experimenting with using libvirt (0.3.3) on a variety of
systems (RHEL, CentOS and Oracle VM). I have run into an issue when
I try to open a local read-only connection to the hypervisor that is
failing only on Oracle VM server release 2.2.0. I have created a root
owned setuid executable that is effectively running as root, but even
so, still cannot open the local read-only connection of the hypervisor.
It only works if I run it directly as root. This is not an option. I
do not understand why it works as is on my RHEL and CentOS machines, but
not my Oracle machine. It would seem as thought it is not checking if
the effective uid is root, just the uid.
A readonly connection essentially just goes to a separate UNIX socket, which
has more relaxed permissions (mode 0777, instead of 0700). The kernel does
the permissions checking when attempting to open it, so it should be using
the effective ID.
Has anyone run into a similar issue or have any suggestions of what I
might try to fix this issue or can tell me that this is a defect that
needs (is) fixed??
I'd suggest trying to strace your process & see where it fails, and/or run
with LIBVIRT_DEBUG=1 environemnt variable set.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5:30 a.m.
On Mon, Jul 12, 2010 at 06:38:27AM -0500, Tavares, John wrote:
Hi Daniel,
Sorry it took so long to back to this. I just recently ran into the
same problem on RHEL 5.5 where I am running into problems using the
root owned setuid executable to get domain information. Basically
all that I needed to do to show this was to modify the example
program dominfo/info1.c that ships with the libvirt source and added
a function that attempts to get any information about that domain.
In this case, I chose the virDomainGetSchedulerType() function. Here
is the output of 3 separate runs, the first will be as a non-root user,
the second is from root and the last is from the same executable but
just root owned setuid and run as a normal user:
I've figured out what is going on here. Originally libvirt provided
a setuid proxy for accessing the Xen driver as non-root. This is
/usr/libexec/libvirt_proxy. This only supports a small sub-set of
API calls and is only for the Xen hypervisor. Later we added a
general purpose daemon (/usr/sbin/libvirtd) that runs as root and
works for all APIs and all hypervisors. This daemon replaces the
setuid proxy, but for reasons of backwards compatability in RHEL5
the proxy is still the default for Xen. The virDomainGetSchedulerType()
API is an example of one that is supported by libvirtd, but not by
the Xen proxy. To force use of the libvirtd, give it an explicit
URI, xen+unix:/// will make it connect to Xen, over libvirtd's
UNIX domain socket.
$ ll info1
-r-xr-xr-x 1 perform perform 9326 Jul 12 16:50 info1
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
This doesn't print schedular info because it is not supported
by the Xen proxy.
$ su
Password:
# ./info1
Domains 0: 2 CPUs
Scheduler = credit
# chown 0:0 info1;chmod 6555 info1
# ll info1
-r-sr-sr-x 1 root root 9326 Jul 12 16:50 info1
# exit
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
libvir: Xen Daemon error : this function is not supported by the hypervisor: unsupported
in xendConfigVersion < 4
Not entirely sure why you get this error message here, but
I strongly recommend against running this setuid regardless.
$ virsh version
error: failed to connect to the hypervisor
You need to pass the --readonly flag to virsh
$ whoami
berrange
$ virsh --readonly version
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
And to make it use the libvirtd instead of the proxy give it a URI:
$ virsh -c xen+unix:/// --readonly schedinfo Domain-0
Scheduler : credit
weight : 256
cap : 0
$ su root -c "virsh version"
Password:
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:03 a.m.
Hi Daniel,
Thank you for looking into this for me and providing such a detailed explanation, I really
appreciate it.
As a related question, will this work for RHEL6 for KVM as well along with the virsh
CLI??
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 6:30 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com; Tomen Tse; Betley, Greg; Chen, Jianjiun
Subject: Re: [libvirt] inability to open local read-only connection
On Mon, Jul 12, 2010 at 06:38:27AM -0500, Tavares, John wrote:
Hi Daniel,
Sorry it took so long to back to this. I just recently ran into the
same problem on RHEL 5.5 where I am running into problems using the
root owned setuid executable to get domain information. Basically
all that I needed to do to show this was to modify the example
program dominfo/info1.c that ships with the libvirt source and added
a function that attempts to get any information about that domain.
In this case, I chose the virDomainGetSchedulerType() function. Here
is the output of 3 separate runs, the first will be as a non-root user,
the second is from root and the last is from the same executable but
just root owned setuid and run as a normal user:
I've figured out what is going on here. Originally libvirt provided
a setuid proxy for accessing the Xen driver as non-root. This is
/usr/libexec/libvirt_proxy. This only supports a small sub-set of
API calls and is only for the Xen hypervisor. Later we added a
general purpose daemon (/usr/sbin/libvirtd) that runs as root and
works for all APIs and all hypervisors. This daemon replaces the
setuid proxy, but for reasons of backwards compatability in RHEL5
the proxy is still the default for Xen. The virDomainGetSchedulerType()
API is an example of one that is supported by libvirtd, but not by
the Xen proxy. To force use of the libvirtd, give it an explicit
URI, xen+unix:/// will make it connect to Xen, over libvirtd's
UNIX domain socket.
$ ll info1
-r-xr-xr-x 1 perform perform 9326 Jul 12 16:50 info1
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
This doesn't print schedular info because it is not supported
by the Xen proxy.
$ su
Password:
# ./info1
Domains 0: 2 CPUs
Scheduler = credit
# chown 0:0 info1;chmod 6555 info1
# ll info1
-r-sr-sr-x 1 root root 9326 Jul 12 16:50 info1
# exit
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
libvir: Xen Daemon error : this function is not supported by the hypervisor: unsupported
in xendConfigVersion < 4
Not entirely sure why you get this error message here, but
I strongly recommend against running this setuid regardless.
$ virsh version
error: failed to connect to the hypervisor
You need to pass the --readonly flag to virsh
$ whoami
berrange
$ virsh --readonly version
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
And to make it use the libvirtd instead of the proxy give it a URI:
$ virsh -c xen+unix:/// --readonly schedinfo Domain-0
Scheduler : credit
weight : 256
cap : 0
$ su root -c "virsh version"
Password:
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:12 a.m.
On Fri, Aug 20, 2010 at 07:03:05AM -0500, Tavares, John wrote:
Hi Daniel,
Thank you for looking into this for me and providing such a detailed explanation, I
really appreciate it.
As a related question, will this work for RHEL6 for KVM as well along with the virsh
CLI??
Yes, just change the URI to 'qemu:///system'. No need for the +unix
bit here because QEMU always goes via libvirtd.
You have KVM in RHEL5.4 or later too that this should work with...
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:28 a.m.
Thanks! This is good to know.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 8:12 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com; Tomen Tse; Betley, Greg; Chen, Jianjiun
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 07:03:05AM -0500, Tavares, John wrote:
Hi Daniel,
Thank you for looking into this for me and providing such a detailed explanation, I
really appreciate it.
As a related question, will this work for RHEL6 for KVM as well along with the virsh
CLI??
Yes, just change the URI to 'qemu:///system'. No need for the +unix
bit here because QEMU always goes via libvirtd.
You have KVM in RHEL5.4 or later too that this should work with...
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:42 a.m.
If you don't mind, I have a few other questions related to this that I'd like to
ask before I forget about them.
1) Is there any technical reason for not shipping the 32-bit version of the libvirt shared
library for 64-bit systems??
2) Is it possible via libvirt to get that some additional information about the guests,
such as:
a) uptime
b) hostname
c) real operating system
Thanks again.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 8:12 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com; Tomen Tse; Betley, Greg; Chen, Jianjiun
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 07:03:05AM -0500, Tavares, John wrote:
Hi Daniel,
Thank you for looking into this for me and providing such a detailed explanation, I
really appreciate it.
As a related question, will this work for RHEL6 for KVM as well along with the virsh
CLI??
Yes, just change the URI to 'qemu:///system'. No need for the +unix
bit here because QEMU always goes via libvirtd.
You have KVM in RHEL5.4 or later too that this should work with...
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
7:58 a.m.
On Fri, Aug 20, 2010 at 07:42:32AM -0500, Tavares, John wrote:
If you don't mind, I have a few other questions related to
this that I'd like to ask before I forget about them.
1) Is there any technical reason for not shipping the 32-bit
version of the libvirt shared library for 64-bit systems??
The libvirt RPC protocol is arch-invariant, so you can use
a 32-bit libvirt library to talk to a 64-bit host, and
vica-verca. So you can install the 32bit library if desired,
but really 32-bit is dead technology, so its better to just
recompile any apps for 64-bit.
2) Is it possible via libvirt to get that some additional
information about the guests, such as:
a) uptime
b) hostname
c) real operating system
You can't reliably get any of that information without having
an agent running inside the guest OS. Currently libvirt does
not concern itself with guest agent services, the focus being
primarily on host OS management. There's a small chance this
may change in the future, but its too early to say for sure.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:15 a.m.
On 08/20/2010 10:58 PM, Daniel P. Berrange wrote:
<snip>
> a) uptime
<snip>
You can't reliably get any of that information without having
an agent running inside the guest OS.
We should be able to get the uptime of the guest's qemu process itself
though, and report that back "over the wire" to a requesting client,
without needing a guest OS agent. Or even keep track of when libvirt
launched the guest as wall clock time.
Not the same thing as "how long has the guest OS been up?", but
potentially useful for management tools.
Regards and best wishes,
Justin Clift
8:16 a.m.
On Fri, Aug 20, 2010 at 11:15:00PM +1000, Justin Clift wrote:
On 08/20/2010 10:58 PM, Daniel P. Berrange wrote:
<snip>
>> a) uptime
<snip>
>You can't reliably get any of that information without having
>an agent running inside the guest OS.
We should be able to get the uptime of the guest's qemu process itself
though, and report that back "over the wire" to a requesting client,
without needing a guest OS agent. Or even keep track of when libvirt
launched the guest as wall clock time.
Yep that would be possible, I was just asuming John wanted the OS uptime
based on his other data points there.
Not the same thing as "how long has the guest OS been up?",
but
potentially useful for management tools.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:49 a.m.
It is available via xm, so my point was if it can, why can't libvirt??
In the end, I had to switch to calling virsh until I can get all of problem resolved.
Ideally, virsh to would be able to provide most of the same data as going thru libvirt
directly. It might also be nice if it provided that data in a more machine parseable
format.
-----Original Message-----
From: Justin Clift [mailto:jclift@redhat.com]
Sent: Friday, August 20, 2010 9:15 AM
To: Daniel P. Berrange
Cc: Tavares, John; libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On 08/20/2010 10:58 PM, Daniel P. Berrange wrote:
<snip>
> a) uptime
<snip>
You can't reliably get any of that information without having
an agent running inside the guest OS.
We should be able to get the uptime of the guest's qemu process itself
though, and report that back "over the wire" to a requesting client,
without needing a guest OS agent. Or even keep track of when libvirt
launched the guest as wall clock time.
Not the same thing as "how long has the guest OS been up?", but
potentially useful for management tools.
Regards and best wishes,
Justin Clift
8:57 a.m.
On Fri, Aug 20, 2010 at 08:49:50AM -0500, Tavares, John wrote:
It is available via xm, so my point was if it can, why can't
libvirt??
No particular reason why not, other than nobody has requested
it before.
NB, 'xm uptime' is giving the uptime of the virtual domain,
not the guest OS running inside it. ie if you reboot the guest
OS, the 'uptime' doesn't get reset because the virtual domain
itself (usually[1]) never stops.
In the end, I had to switch to calling virsh until I can get
all of problem resolved.
Ideally, virsh to would be able to provide most of the same
data as going thru libvirt directly. It might also be nice
if it provided that data in a more machine parseable format.
We do plan to make it available in machine readable formats
in the future, but in most cases you'd be better off using
one of the APIs directly instead of virsh. There's a choice
of C, python, perl, java, ocaml, ruby. It would be significantly
more efficient to use the API becasue you would be avoiding the
overhead of opening & closing a connection for every single
data item collected.
Daniel
[1] For paravirt guests IIRC Xen may actually destroy & recreate
the virtual domain during a reboot, but fullvirt should be
transparent.
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:03 a.m.
I initially used libvirt directly, but when I started running into the lack of a 32-bit
library on 64-bit issue and the inconsistent effective root user behavior issue, I needed
to abandon the use of it and turned to virsh as it had no such issues and pay the
performance penalty for stability. Maybe for our next release I will go back to it, but
it is a little late in the game now for me to go back now.
It also looks like the default version that comes with each revision of RHEL is a bit
dated.
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 9:58 AM
To: Tavares, John
Cc: Justin Clift; libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 08:49:50AM -0500, Tavares, John wrote:
It is available via xm, so my point was if it can, why can't
libvirt??
No particular reason why not, other than nobody has requested
it before.
NB, 'xm uptime' is giving the uptime of the virtual domain,
not the guest OS running inside it. ie if you reboot the guest
OS, the 'uptime' doesn't get reset because the virtual domain
itself (usually[1]) never stops.
In the end, I had to switch to calling virsh until I can get
all of problem resolved.
Ideally, virsh to would be able to provide most of the same
data as going thru libvirt directly. It might also be nice
if it provided that data in a more machine parseable format.
We do plan to make it available in machine readable formats
in the future, but in most cases you'd be better off using
one of the APIs directly instead of virsh. There's a choice
of C, python, perl, java, ocaml, ruby. It would be significantly
more efficient to use the API becasue you would be avoiding the
overhead of opening & closing a connection for every single
data item collected.
Daniel
[1] For paravirt guests IIRC Xen may actually destroy & recreate
the virtual domain during a reboot, but fullvirt should be
transparent.
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:15 a.m.
Should the memory utilization as report from inside a guest match that as reported by the
hypervisor?? I think one of my QA engineers may have pointed out a Windows guest where
this was not the case.
-----Original Message-----
From: Tavares, John
Sent: Friday, August 20, 2010 10:04 AM
To: 'Daniel P. Berrange'
Cc: Justin Clift; libvir-list(a)redhat.com
Subject: RE: [libvirt] inability to open local read-only connection
I initially used libvirt directly, but when I started running into the lack of a 32-bit
library on 64-bit issue and the inconsistent effective root user behavior issue, I needed
to abandon the use of it and turned to virsh as it had no such issues and pay the
performance penalty for stability. Maybe for our next release I will go back to it, but
it is a little late in the game now for me to go back now.
It also looks like the default version that comes with each revision of RHEL is a bit
dated.
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 9:58 AM
To: Tavares, John
Cc: Justin Clift; libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 08:49:50AM -0500, Tavares, John wrote:
It is available via xm, so my point was if it can, why can't
libvirt??
No particular reason why not, other than nobody has requested
it before.
NB, 'xm uptime' is giving the uptime of the virtual domain,
not the guest OS running inside it. ie if you reboot the guest
OS, the 'uptime' doesn't get reset because the virtual domain
itself (usually[1]) never stops.
In the end, I had to switch to calling virsh until I can get
all of problem resolved.
Ideally, virsh to would be able to provide most of the same
data as going thru libvirt directly. It might also be nice
if it provided that data in a more machine parseable format.
We do plan to make it available in machine readable formats
in the future, but in most cases you'd be better off using
one of the APIs directly instead of virsh. There's a choice
of C, python, perl, java, ocaml, ruby. It would be significantly
more efficient to use the API becasue you would be avoiding the
overhead of opening & closing a connection for every single
data item collected.
Daniel
[1] For paravirt guests IIRC Xen may actually destroy & recreate
the virtual domain during a reboot, but fullvirt should be
transparent.
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:30 a.m.
On Fri, Aug 20, 2010 at 09:15:04AM -0500, Tavares, John wrote:
Should the memory utilization as report from inside a guest match
that
as reported by the hypervisor?? I think one of my QA engineers may have
pointed out a Windows guest where this was not the case.
It depends what libvirt APi you are referring to when you talk about
memory utilization. virDomainGetInfo returns data about the memory
allocated to the guest by the hypervisor. This says nothing about
what the guest OS is doing with the memory. eg, the guest has been
given 500 MB, but its OS may have only used 200 MB so far.
We have a fairly new API virDomainGetMemoryStats that can give info
about how much of its allocated memory a guest has actually used,
along with stats on swap in/out and major/minor fails. This is only
implemented for KVM + virtio though and requires a guest with a pretty
new balloon driver to report the data
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:34 a.m.
When I was using libvirt directly, I did see that, but could never make it work. This is
some of the additional memory/performance information we would be interested in. In the
event I stick with using virsh, it is available it as well as I have yet to see it??
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 10:30 AM
To: Tavares, John
Cc: Justin Clift; libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 09:15:04AM -0500, Tavares, John wrote:
Should the memory utilization as report from inside a guest match
that
as reported by the hypervisor?? I think one of my QA engineers may have
pointed out a Windows guest where this was not the case.
It depends what libvirt APi you are referring to when you talk about
memory utilization. virDomainGetInfo returns data about the memory
allocated to the guest by the hypervisor. This says nothing about
what the guest OS is doing with the memory. eg, the guest has been
given 500 MB, but its OS may have only used 200 MB so far.
We have a fairly new API virDomainGetMemoryStats that can give info
about how much of its allocated memory a guest has actually used,
along with stats on swap in/out and major/minor fails. This is only
implemented for KVM + virtio though and requires a guest with a pretty
new balloon driver to report the data
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:48 a.m.
On Fri, Aug 20, 2010 at 09:34:22AM -0500, Tavares, John wrote:
When I was using libvirt directly, I did see that, but could
never make it work. This is some of the additional memory/
performance information we would be interested in. In the
event I stick with using virsh, it is available it as well
as I have yet to see it??
The 'virsh dommemstat' command is the one you want. It will only
work on a RHEL-6 host, with a RHEL6 / F14 guests eg
# virsh dommemstat f14x8_64
swap_in 0
swap_out 0
major_fault 534
minor_fault 714956
unused 595656
available 1023720
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
9:53 a.m.
Thanks again Daniel.
All of this information is going to be very handy as we are going to start to look at KVM
and RHEL6 soon.
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 10:48 AM
To: Tavares, John
Cc: Justin Clift; libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 09:34:22AM -0500, Tavares, John wrote:
When I was using libvirt directly, I did see that, but could
never make it work. This is some of the additional memory/
performance information we would be interested in. In the
event I stick with using virsh, it is available it as well
as I have yet to see it??
The 'virsh dommemstat' command is the one you want. It will only
work on a RHEL-6 host, with a RHEL6 / F14 guests eg
# virsh dommemstat f14x8_64
swap_in 0
swap_out 0
major_fault 534
minor_fault 714956
unused 595656
available 1023720
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:47 a.m.
You may consider 32-bit a dead technology, but we do not. We have one 32-bit application
that runs and collects configuration and performance data for all types of Linux based
system regardless if it is 32-bit or 64-bit. Until such time as there is no 32-bit
systems, I do not see this going away anytime soon.
I can understand that libvirt is not too concerned about the internal of the guest, but
for our purposes, it is a concern. While having the uptime (which is available from xm)
and hostname (a nice to have), the reporting of the real operating system without forcing
the user to deploy agents into each and every guest just to obtain this sort of basic
information about it. The operating system is of importance to us for use cases like
server/workload consolidation where you need to know if the workload from one system can
indeed be migrated to another, where the first factor is are they running the same
operating systems.
This is something to keep in mind.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Friday, August 20, 2010 8:58 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com; Tomen Tse; Betley, Greg; Chen, Jianjiun
Subject: Re: [libvirt] inability to open local read-only connection
On Fri, Aug 20, 2010 at 07:42:32AM -0500, Tavares, John wrote:
If you don't mind, I have a few other questions related to
this that I'd like to ask before I forget about them.
1) Is there any technical reason for not shipping the 32-bit
version of the libvirt shared library for 64-bit systems??
The libvirt RPC protocol is arch-invariant, so you can use
a 32-bit libvirt library to talk to a 64-bit host, and
vica-verca. So you can install the 32bit library if desired,
but really 32-bit is dead technology, so its better to just
recompile any apps for 64-bit.
2) Is it possible via libvirt to get that some additional
information about the guests, such as:
a) uptime
b) hostname
c) real operating system
You can't reliably get any of that information without having
an agent running inside the guest OS. Currently libvirt does
not concern itself with guest agent services, the focus being
primarily on host OS management. There's a small chance this
may change in the future, but its too early to say for sure.
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
8:53 a.m.
On 08/20/2010 06:42 AM, Tavares, John wrote:
2) Is it possible via libvirt to get that some additional information about the guests,
such as:
a) uptime
b) hostname
c) real operating system
hostname and OS of local disk images can be determined using libguestfs
(in particular, I think virt-inspector command line tool from the
libguestfs suite answers both of those questions). I'm not sure if
uptime can also be retrieved with libguestfs, but if not, it is a
reasonable feature request.
--
Eric Blake eblake(a)redhat.com +1-801-349-2682
Libvirt virtualization library http://libvirt.org
3:49 p.m.
Hi Daniel,
I added some more info to this. I added a call to try to get the max number of vcpus as
well. For starters, here is the output of my 3 tests:
$ info1
-bash: info1: command not found
[perform@pun-rlnx55-bpa01 tavares]$ ./info1
Failed to bind to socket /tmp/livirt_proxy_conn
Domains 0: 2 CPUs
Failed to get scheduler inforamtion for Domain 0
Failed to get max vcpus for Domain 0
$ su root -c "./info1"
Password:
Domains 0: 2 CPUs
Scheduler = credit
Max VCPUs = 2
$ su root -c "chown 0:0 info1;chmod 6555 info1"
Password:
[perform@pun-rlnx55-bpa01 tavares]$ ./info1
Domains 0: 2 CPUs
libvir: Xen Daemon error : this function is not supported by the hypervisor: unsupported
in xendConfigVersion < 4
Failed to get scheduler inforamtion for Domain 0
Failed to get max vcpus for Domain 0
I have attached a zipped tar file which now contains the source code, the executable, the
strace output from all three tests and the configuration files from /etc/libvirt. Please
let me know if there is anything else that I can provide. I will also see if I could do
this on the Oracle VM Dom0 (if I can remember which one I used) and run similar tests.
Thanks.
John
-----Original Message-----
From: Tavares, John
Sent: Monday, July 12, 2010 7:38 AM
To: 'Daniel P. Berrange'
Cc: libvir-list(a)redhat.com; 'Tomen Tse'; Betley, Greg; Chen, Jianjiun; Tavares,
John
Subject: RE: [libvirt] inability to open local read-only connection
Importance: High
Hi Daniel,
Sorry it took so long to back to this. I just recently ran into the same problem on RHEL
5.5 where I am running into problems using the root owned setuid executable to get domain
information. Basically all that I needed to do to show this was to modify the example
program dominfo/info1.c that ships with the libvirt source and added a function that
attempts to get any information about that domain. In this case, I chose the
virDomainGetSchedulerType() function. Here is the output of 3 separate runs, the first
will be as a non-root user, the second is from root and the last is from the same
executable but just root owned setuid and run as a normal user:
$ ll info1
-r-xr-xr-x 1 perform perform 9326 Jul 12 16:50 info1
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
$ su
Password:
# ./info1
Domains 0: 2 CPUs
Scheduler = credit
# chown 0:0 info1;chmod 6555 info1
# ll info1
-r-sr-sr-x 1 root root 9326 Jul 12 16:50 info1
# exit
$ whoami
perform
$ ./info1
Domains 0: 2 CPUs
libvir: Xen Daemon error : this function is not supported by the hypervisor: unsupported
in xendConfigVersion < 4
$ virsh version
error: failed to connect to the hypervisor
$ su root -c "virsh version"
Password:
Compiled against library: libvir 0.6.3
Using library: libvir 0.6.3
Using API: Xen 3.0.1
Running hypervisor: Xen 3.1.0
As you can see, even virsh needs to be run as root in order to connect to the hypervisor.
To help, I have attached both the source code and the executable I was using. Please let
me know if anything else is required to get this problem resolved.
Thanks.
John
-----Original Message-----
From: Daniel P. Berrange [mailto:berrange@redhat.com]
Sent: Monday, March 01, 2010 7:58 AM
To: Tavares, John
Cc: libvir-list(a)redhat.com
Subject: Re: [libvirt] inability to open local read-only connection
On Thu, Feb 25, 2010 at 02:34:02PM -0600, Tavares, John wrote:
I have been experimenting with using libvirt (0.3.3) on a variety of
systems (RHEL, CentOS and Oracle VM). I have run into an issue when
I try to open a local read-only connection to the hypervisor that is
failing only on Oracle VM server release 2.2.0. I have created a root
owned setuid executable that is effectively running as root, but even
so, still cannot open the local read-only connection of the hypervisor.
It only works if I run it directly as root. This is not an option. I
do not understand why it works as is on my RHEL and CentOS machines, but
not my Oracle machine. It would seem as thought it is not checking if
the effective uid is root, just the uid.
A readonly connection essentially just goes to a separate UNIX socket, which
has more relaxed permissions (mode 0777, instead of 0700). The kernel does
the permissions checking when attempting to open it, so it should be using
the effective ID.
Has anyone run into a similar issue or have any suggestions of what I
might try to fix this issue or can tell me that this is a defect that
needs (is) fixed??
I'd suggest trying to strace your process & see where it fails, and/or run
with LIBVIRT_DEBUG=1 environemnt variable set.
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5:31 a.m.
On Mon, Jul 12, 2010 at 03:49:17PM -0500, Tavares, John wrote:
Hi Daniel,
I added some more info to this. I added a call to try to get the max number of vcpus as
well. For starters, here is the output of my 3 tests:
$ info1
-bash: info1: command not found
[perform@pun-rlnx55-bpa01 tavares]$ ./info1
Failed to bind to socket /tmp/livirt_proxy_conn
Domains 0: 2 CPUs
Failed to get scheduler inforamtion for Domain 0
Failed to get max vcpus for Domain 0
If you edit your demo program to pass 'xen+unix:///' to
virConnectOpenReadonly() this should work as you expect
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
5021
days inactive
5197
days old
23 comments
4 participants
participants (4)
-
Daniel P. Berrange -
Eric Blake -
Justin Clift -
Tavares, John