[libvirt] libvirt / libxl testing emails
Ian / Ian, Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list. I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions. -George
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions.
I guess from my POV the answer probably hinges on the reliability of the test harness. If it has a very low false failure rate, then it'd be ok to send it to this list. Conversely if there's a non-trivial false failure rate it'd be better to have it go to an individual who filters it and just forwards the real failures to the list. An alternative idea would be for us to setup a new 'libvirt-testing' mailing list, exclusively for spamming by CI systems, that people could opt-in to subscribing to. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
On 06/03/2014 03:30 PM, Daniel P. Berrange wrote:
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions. I guess from my POV the answer probably hinges on the reliability of the test harness. If it has a very low false failure rate, then it'd be ok to send it to this list. Conversely if there's a non-trivial false failure rate it'd be better to have it go to an individual who filters it and just forwards the real failures to the list.
An alternative idea would be for us to setup a new 'libvirt-testing' mailing list, exclusively for spamming by CI systems, that people could opt-in to subscribing to.
In my experience the test e-mails are fairly chatty, and fail for any number of reasons. The most recent test mail can be found here, for instance: http://www.gossamer-threads.com/lists/xen/devel/332583 And it seems to have failed because of a problem with the ARM testing hardware (since builds of Xen and Linux fail as well). My suspicion is that most people on libvir-list will learn to ignore the mails unless someone raises a particular failure as something worth looking at; in which case it might be better to have a separate opt-in list. (Or to wait until someone on xen-devel raises the issue.) -George
On Tue, 2014-06-03 at 15:37 +0100, George Dunlap wrote:
On 06/03/2014 03:30 PM, Daniel P. Berrange wrote:
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions. I guess from my POV the answer probably hinges on the reliability of the test harness. If it has a very low false failure rate, then it'd be ok to send it to this list. Conversely if there's a non-trivial false failure rate it'd be better to have it go to an individual who filters it and just forwards the real failures to the list.
An alternative idea would be for us to setup a new 'libvirt-testing' mailing list, exclusively for spamming by CI systems, that people could opt-in to subscribing to.
In my experience the test e-mails are fairly chatty, and fail for any number of reasons. The most recent test mail can be found here, for instance:
http://www.gossamer-threads.com/lists/xen/devel/332583
And it seems to have failed because of a problem with the ARM testing hardware (since builds of Xen and Linux fail as well).
My suspicion is that most people on libvir-list will learn to ignore the mails unless someone raises a particular failure as something worth looking at; in which case it might be better to have a separate opt-in list. (Or to wait until someone on xen-devel raises the issue.)
The majority of the failures so far have been infrastructure issues, so I expect you are right. I tend to keep an eye on this particular flight and I'm happy to forward anything which looks like an upstream issue (it's only happened once so far). I'm equally happy to add some list or individual other to the config for places to spam with the results. Jim was happy to keep getting them via xen-devel. Ian.
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions.
Actually Dan Berrange should be included. I think the nicest would be a private ping and the information available online, I hink that hw our existing rebuilder machines works but I would let Dan suggest the best way at this point. thanks for getting back to me, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/
On Wed, 2014-06-04 at 00:00 +0800, Daniel Veillard wrote:
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions.
Actually Dan Berrange should be included. I think the nicest would be a private ping and the information available online,
Why private and not to the libvir list? The information is available online via the test report (which don't stay around forever, but for weeks at least). I'm happy to do whatever you guys would prefer, just let me know. Ian.
On Fri, Jun 06, 2014 at 09:25:46AM +0100, Ian Campbell wrote:
On Wed, 2014-06-04 at 00:00 +0800, Daniel Veillard wrote:
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions.
Actually Dan Berrange should be included. I think the nicest would be a private ping and the information available online,
Why private and not to the libvir list?
Well the point is that the libvir-list is already high traffic and avoiding automated mails to it would help. Also the compile box doesn't send mail directly to the list in case of breakage either, so that would be coherent.
The information is available online via the test report (which don't stay around forever, but for weeks at least).
I'm happy to do whatever you guys would prefer, just let me know.
Do you have a pointer to where the test reports sit, that would be helpful as a first step :-) thanks, Daniel -- Daniel Veillard | Open Source and Standards, Red Hat veillard@redhat.com | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ http://veillard.com/ | virtualization library http://libvirt.org/
On Fri, 2014-06-06 at 17:32 +0800, Daniel Veillard wrote:
On Fri, Jun 06, 2014 at 09:25:46AM +0100, Ian Campbell wrote:
On Wed, 2014-06-04 at 00:00 +0800, Daniel Veillard wrote:
On Tue, Jun 03, 2014 at 03:16:49PM +0100, George Dunlap wrote:
Ian / Ian,
Daniel V. was lurking on a recent CentOS Virt SIG IRC meetings, and asked about testing the libvirt git repo against libxl. I mentioned that we already had some basic tests to do that, and he asked whether it would make sense to have the push gate e-mail the libvirt list.
I have no idea how the libvirt people feel about that, so I'll let DV make his case, and also let he Ians give their opinions.
Actually Dan Berrange should be included. I think the nicest would be a private ping and the information available online,
Why private and not to the libvir list?
Well the point is that the libvir-list is already high traffic and avoiding automated mails to it would help.
Oh, I thought you were referring to my manual CC to libvirt that I have been doing when an issue looked to be upstream related (exactly once so far). Yes, I certainly wouldn't recommend sending our automated test reports to the list. I'm happy to add an automated CC to whatever mailbox you guys want.
Also the compile box doesn't send mail directly to the list in case of breakage either, so that would be coherent.
The information is available online via the test report (which don't stay around forever, but for weeks at least).
I'm happy to do whatever you guys would prefer, just let me know.
Do you have a pointer to where the test reports sit, that would be helpful as a first step :-)
There is a link in each test report e.g. http://lists.xen.org/archives/html/xen-devel/2014-06/msg00776.html links to http://www.chiark.greenend.org.uk/~xensrcts/logs/26970/ note that the logs are expired after a couple of weeks (this one is still live though). http://xenbits.xensource.com/gitweb/?p=osstest.git;a=blob;f=README;h=9a85549... contains some terminology etc which might help you make sense of it. Currently these are only build tests, because we've not taught our test system about virsh yet, hence the "guest-start" step is marked as "never passed". By contrast http://lists.xen.org/archives/html/xen-devel/2014-06/msg00557.html http://www.chiark.greenend.org.uk/~xensrcts/logs/26947/ is a failed test (caused by a network glitch on our end) http://lists.xen.org/archives/html/xen-devel/2014-05/msg00528.html has some useful background one which test flights (flight == the FOO in the "[FOO test]" but of the email subject) include libvirt tests and what they are testing against what (summary: libvirt flight is new libvirt against already tested xen and xen-* flights are new xen from that branch against already an tested libvirt). Ian.
participants (4)
-
Daniel P. Berrange -
Daniel Veillard -
George Dunlap -
Ian Campbell