[Libvir] ANNOUNCE: Virt-top - a top-like utility for displaying virtualization stats
I'm pleased to announce the first release of virt-top, which is a top-like utility for displaying virtualization stats. It aims to look and feel very much like regular 'top', so as to be as familiar as possible for systems administrators. You can also use it as a pleasant replacement for xentop. It uses libvirt, so can display stats across a variety of different hypervisors and virtualization systems (not just Xen, although that is where the testing has gone so far). http://et.redhat.com/~rjones/virt-top/ The license is a combination of LGPL (for the library) and GPL (for the virt-top program). Current status -------------- You can view domains and use familiar keys like 'P'/'M'/... to sort by processor/memory/..., and 'd'/'s' to set the delay between updates. Also some common top command-line options are implemented. The man page is here: http://et.redhat.com/~rjones/virt-top/virt-top.txt There are a variety of source and binary RPMs available for Fedora users. I don't yet have a working Debian/Ubuntu package, but will have a go at making one tomorrow. The next thing I'll be working on is showing virtual and physical CPU usage of guests. After that I'm hoping to discuss extensions to libvirt to make other interesting statistics available to virt-top, in particular disk and network I/O stats. There are a few data collection artifacts which need to be investigated. In particular, %CPU sometimes goes over 100%. Obviously accurate data collection is an important goal for this tool. Memory usage is good: typical 'RES' (in regular top) for virt-top is just under 2.5 MB, and I've had it running for hours at a time without memory usage increasing, which seems to indicate that there aren't any major memory leaks. Development ----------- The program is currently very small: exactly 500 lines of code! If you want to dive in and send me patches they are most welcome, but remember that I'm trying to make this utility act as much like 'top' as possible, so if 'top' does it in a particular way, then I'd prefer virt-top to do the same thing. (May not apply to obscure top functionality, if top does something silly). If you're a systems administrator, not a programmer, and you'd like virt-top to have some particular feature, then please let me know, and I will be happy to review it and code it for you. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Mon, Aug 06, 2007 at 10:15:48PM +0100, Richard W.M. Jones wrote:
There are a few data collection artifacts which need to be investigated. In particular, %CPU sometimes goes over 100%. Obviously accurate data collection is an important goal for this tool.
AFAICT it is impossible to stop it going over 100% - we have same issue in virt-manager when calculating CPU usage. To caculate the % usage you are looking at the differential between CPU time from two calls to virDomainGetInfo vs the time period between two gettimeofday() calls. If the time delta between your gettimeofday() calls does not exactly match the time delta between your virDomainGetInfo() calls, and the domain you're measuring is very active then you can end up calcuating
100% occassionally. Not by much - typically < 1% over, but I don't really see any way around it.
Regards, Dan. -- |=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=| |=- Perl modules: http://search.cpan.org/~danberr/ -=| |=- Projects: http://freshmeat.net/~danielpb/ -=| |=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|
Hi Richard, I haven't followed development of libvirt for a while since it was unuseable for our monitoring due to the inaccurate measuring of cpu%. At that time I was happy to have a working lib in version 0.1.9. When you came up with vir-top, I wanted to give it a try. But therefore at least version 0.3.1 of libvirt is needed. So I had to compile the library again and run in the following error during make: <error> remote_internal.c: At top level: remote_internal.c:969: warning: unused parameter 'sock' make[2]: *** [libvirt_la-remote_internal.lo] Error 1 make[2]: Leaving directory `/opt/libvirt-0.3.1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/opt/libvirt-0.3.1' make: *** [all] Error 2 </error> I tried to configure without remote, but had no luck. Could you be so nice and tell me which requesitions my systems has to fulfill that I can successfully compile libvirt? By the way. I'm using a RHES 4.x based distribution, Xen 3.0.3 and kernel 2.6.18 in dom0. I have only Xen installed an only use libvirt on top of Xen. Thanks for your help, Jan On 06.08.2007, at 23:15, Richard W.M. Jones wrote:
I'm pleased to announce the first release of virt-top, which is a top-like utility for displaying virtualization stats.
It aims to look and feel very much like regular 'top', so as to be as familiar as possible for systems administrators. You can also use it as a pleasant replacement for xentop.
It uses libvirt, so can display stats across a variety of different hypervisors and virtualization systems (not just Xen, although that is where the testing has gone so far).
http://et.redhat.com/~rjones/virt-top/
The license is a combination of LGPL (for the library) and GPL (for the virt-top program).
Current status --------------
You can view domains and use familiar keys like 'P'/'M'/... to sort by processor/memory/..., and 'd'/'s' to set the delay between updates. Also some common top command-line options are implemented. The man page is here: http://et.redhat.com/~rjones/ virt-top/virt-top.txt
There are a variety of source and binary RPMs available for Fedora users. I don't yet have a working Debian/Ubuntu package, but will have a go at making one tomorrow.
The next thing I'll be working on is showing virtual and physical CPU usage of guests. After that I'm hoping to discuss extensions to libvirt to make other interesting statistics available to virt- top, in particular disk and network I/O stats.
There are a few data collection artifacts which need to be investigated. In particular, %CPU sometimes goes over 100%. Obviously accurate data collection is an important goal for this tool.
Memory usage is good: typical 'RES' (in regular top) for virt-top is just under 2.5 MB, and I've had it running for hours at a time without memory usage increasing, which seems to indicate that there aren't any major memory leaks.
Development -----------
The program is currently very small: exactly 500 lines of code!
If you want to dive in and send me patches they are most welcome, but remember that I'm trying to make this utility act as much like 'top' as possible, so if 'top' does it in a particular way, then I'd prefer virt-top to do the same thing. (May not apply to obscure top functionality, if top does something silly).
If you're a systems administrator, not a programmer, and you'd like virt-top to have some particular feature, then please let me know, and I will be happy to review it and code it for you.
Rich.
-- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903 <virt-top.png> -- Libvir-list mailing list Libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Jan Michael wrote:
Hi Richard,
I haven't followed development of libvirt for a while since it was unuseable for our monitoring due to the inaccurate measuring of cpu%. At that time I was happy to have a working lib in version 0.1.9.
When you came up with vir-top, I wanted to give it a try. But therefore at least version 0.3.1 of libvirt is needed. So I had to compile the library again and run in the following error during make:
<error> remote_internal.c: At top level: remote_internal.c:969: warning: unused parameter 'sock' make[2]: *** [libvirt_la-remote_internal.lo] Error 1 make[2]: Leaving directory `/opt/libvirt-0.3.1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/opt/libvirt-0.3.1' make: *** [all] Error 2 </error>
I tried to configure without remote, but had no luck.
That's a really strange error. I just downloaded and compiled 0.3.1 here with no problem, so it may be something to do with the version of gnutls you are using. Can you send me the complete output of ./configure and make please.
Could you be so nice and tell me which requesitions my systems has to fulfill that I can successfully compile libvirt?
By the way. I'm using a RHES 4.x based distribution, Xen 3.0.3 and kernel 2.6.18 in dom0. I have only Xen installed an only use libvirt on top of Xen.
If you're on x86-64 then the binary (http://www.annexia.org/tmp/virt-top/virt-top-0.3.1.2-1.fc8.x86_64.rpm) should just work for you. If you're on another arch, or if it doesn't work, then I can install RHEL 4 ES here and guide you through the build process - I've only tried it on RHEL 5 / F7 myself. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Hi Richard, On 07.08.2007, at 10:40, Richard W.M. Jones wrote:
That's a really strange error. I just downloaded and compiled 0.3.1 here with no problem, so it may be something to do with the version of gnutls you are using.
I have installed these rpm packages: gnutls-devel-1.0.20-3.2.3 gnutls-1.0.20-3.2.3 In consideration of the version Daniel is using, these ones are fairly old, but the latest available for my distribution (Scientific Linux)
Can you send me the complete output of ./configure and make please.
Sure! Please find them attached to this mail.
If you're on x86-64 then the binary (http://www.annexia.org/tmp/ virt-top/virt-top-0.3.1.2-1.fc8.x86_64.rpm) should just work for you. If you're on another arch, or if it doesn't work, then I can install RHEL 4 ES here and guide you through the build process - I've only tried it on RHEL 5 / F7 myself.
I have also a 64-bit machine installed with Scientific Linux 4.5. On that machine I tried to install the rpm mentioned by you: <rpm log> [root@pcitfiot08 opt]# rpm -ivh virt-top-0.3.1.2-1.fc8.x86_64.rpm error: Failed dependencies: libc.so.6(GLIBC_2.4)(64bit) is needed by virt- top-0.3.1.2-1.fc8.x86_64 libtinfo.so.5()(64bit) is needed by virt- top-0.3.1.2-1.fc8.x86_64 libvirt.so.0()(64bit) is needed by virt- top-0.3.1.2-1.fc8.x86_64 <rpm log> I have no glibc 2.4 rpm package which is supported by our linux support. The same problem is with ncurses. If I understand things right then libtinfo is included in ncurses >= 5.6. We are still using 5.4. No newer package is available. Is my distro to old for libvirt > 1.9 and vir-top? Or ist there any chance to get those things working without updating the whole system? Thanks, Jan
Jan Michael wrote:
I have no glibc 2.4 rpm package which is supported by our linux support. The same problem is with ncurses. If I understand things right then libtinfo is included in ncurses >= 5.6. We are still using 5.4. No newer package is available.
Is my distro to old for libvirt > 1.9 and vir-top? Or ist there any chance to get those things working without updating the whole system?
No your distro isn't too old. I'm downloading RHEL 4 now so that I can test building on this platform. Should have something tomorrow. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Jan Michael wrote:
Hi Richard,
On 07.08.2007, at 10:40, Richard W.M. Jones wrote:
That's a really strange error. I just downloaded and compiled 0.3.1 here with no problem, so it may be something to do with the version of gnutls you are using.
I have installed these rpm packages:
gnutls-devel-1.0.20-3.2.3 gnutls-1.0.20-3.2.3
In consideration of the version Daniel is using, these ones are fairly old, but the latest available for my distribution (Scientific Linux)
Can you send me the complete output of ./configure and make please.
Sure! Please find them attached to this mail.
The error was slightly more than you showed in the first email: remote_internal.c:63: error: syntax error before "gnutls_session_t" remote_internal.c:63: warning: no semicolon at end of struct or union [etc] It turns out that your version of gnutls (1.0.20) didn't have gnutls_session_t, seems it was called gnutls_session_int instead. I'm going to add some tests to our configure script to detect ancient versions of gnutls and refuse to compile. If you can upgrade to a more recent version, please do. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
Richard W.M. Jones wrote:
remote_internal.c:63: error: syntax error before "gnutls_session_t" remote_internal.c:63: warning: no semicolon at end of struct or union [etc]
It turns out that your version of gnutls (1.0.20) didn't have gnutls_session_t, seems it was called gnutls_session_int instead. I'm going to add some tests to our configure script to detect ancient versions of gnutls and refuse to compile. If you can upgrade to a more recent version, please do.
Gah - actually I have the same problem on Debian. I'll take a look at this version of gnutls and see if we can use it instead. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
On Tue, Aug 07, 2007 at 10:08:21AM +0200, Jan Michael wrote:
Hi Richard,
I haven't followed development of libvirt for a while since it was unuseable for our monitoring due to the inaccurate measuring of cpu%. At that time I was happy to have a working lib in version 0.1.9.
When you came up with vir-top, I wanted to give it a try. But therefore at least version 0.3.1 of libvirt is needed. So I had to compile the library again and run in the following error during make:
<error> remote_internal.c: At top level: remote_internal.c:969: warning: unused parameter 'sock' make[2]: *** [libvirt_la-remote_internal.lo] Error 1 make[2]: Leaving directory `/opt/libvirt-0.3.1/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/opt/libvirt-0.3.1' make: *** [all] Error 2 </error>
Hum, I lack context here to analyze the problem. To me sock is declared as a parameter for negotiate_gnutls_on_connection() line 969 and used in gnutls_transport_set_ptr() call line 1011 in that function. So it looks used to me unless for some weird reason gnutls_transport_set_ptr is remapped to a macro not using its argument. Also it is a warning. A warning should not stop compilation with an error. So I really don't understand the data you provided.
I tried to configure without remote, but had no luck.
Could you be so nice and tell me which requesitions my systems has to fulfill that I can successfully compile libvirt?
here I have gnutls-devel-1.4.1-2
By the way. I'm using a RHES 4.x based distribution, Xen 3.0.3 and kernel 2.6.18 in dom0. I have only Xen installed an only use libvirt on top of Xen.
okay, thanks, Daniel -- Red Hat Virtualization group http://redhat.com/virtualization/ Daniel Veillard | virtualization library http://libvirt.org/ veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
Jan Michael wrote:
Could you be so nice and tell me which requesitions my systems has to fulfill that I can successfully compile libvirt?
By the way. I'm using a RHES 4.x based distribution, Xen 3.0.3 and kernel 2.6.18 in dom0. I have only Xen installed an only use libvirt on top of Xen.
You might want to try the Debian binary I built, which you can now find on this page: http://et.redhat.com/~rjones/virt-top/ It may or may not work with your old libvirt - you'll have to try it. But it is most likely to work if you can compile libvirt from CVS with the patch that I just posted to this list. Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
There is a significantly updated version of virt-top now available. On systems which support it, you can show the mapping of domains to physical CPUs and the amount of each physical CPU being used. Screenshots: http://et.redhat.com/~rjones/virt-top/screenshots.html There are also binaries available to download for i386 and x86-64. (The binaries are just self-contained Linux executables, no extra files or dependencies are needed, except for libvirt.so). Downloads: http://et.redhat.com/~rjones/virt-top/download.html If you are a systems administrator, what do you think could be improved about virt-top? Try it out and let me have your suggestions ... Rich. -- Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/ Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 03798903
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
Jan Michael -
Richard W.M. Jones