On 08/13/2012 10:19 AM, Eric Blake wrote:

On 08/13/2012 08:04 AM, Cole Robinson wrote:

...

On 08/13/2012 09:59 AM, Daniel P. Berrange wrote:

...

On Mon, Aug 13, 2012 at 09:56:49AM -0400, Cole Robinson wrote:

...

I just pushed a bunch of patches to -maint branches, but both are giving me distcheck errors that seem related to a gnulib update:

ERROR: files left in build directory after distclean: ./.sc-start-sc_vulnerable_makefile_CVE-2012-3386

'make maintainer-clean' doesn't clean up leftover .sc-start files from one failed syntax check, even if you resolve the syntax check in the meantime. I don't know why the .sc file is getting bundled into the tarball, but it should not be; and that would possibly explain the error you are seeing. There may be further improvements needed on gnulib's side (hence the cc), but for now, I think the workaround for you is to just manually delete that .sc-start* file prior to running distcheck.

Found the root of the issue - it's libvirt's fault. Gnulib's maint.mk takes the initial definition of local-checks-to-skip, and from that, creates a macro 'local-checks' using a := rule: local-check := \ $(patsubst sc_%, sc_%.z, \ $(filter-out $(local-checks-to-skip), $(local-checks-available))) But libvirt's cfg.mk is conditionally running the local-checks-to-skip rule, via: # Most developers don't run 'make distcheck'. We want the official # dist to be secure, but don't want to penalize other developers # using a distro that has not yet picked up the automake fix. # FIXME remove this ifeq (making the syntax check unconditional) # once fixed automake (1.11.6 or 1.12.2+) is more common. ifeq ($(filter dist%, $(MAKECMDGOALS)), ) local-checks-to-skip += sc_vulnerable_makefile_CVE-2012-3386 else distdir: sc_vulnerable_makefile_CVE-2012-3386 endif Because distdir depends on the full sc_ name, rather than the sc_.z rewrite, maint.mk's timing rules don't get properly run, so the .sc-start-* file doesn't get cleaned up. I think with a bit more tweaking to libvirt's cfg.mk, I can get this working again. Meanwhile, would gnulib like to incorporate this hack from libvirt? After all, the current Automake vulnerability only affects you if you run 'make dist' or 'make distcheck'; it does not impact normal day-to-day development. Therefore, running the syntax check only in the vulnerable cases, and in such a way that the syntax check stops make before the vulnerability can actually be triggered, without penalizing day-to-day development for people relying on their distro rather than using a hand-built automake, seems like it would be nice to share among multiple packages. [It's a shame that more than a month after the CVE was reported and both Fedora 17 and RHEL 6.3 are still vulnerable, but that's a story for another day.] -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org

On Mon, Aug 13, 2012 at 10:23:39AM -0600, Eric Blake wrote:

On 08/13/2012 07:56 AM, Cole Robinson wrote:

...

I tried to reproduce on master, but it has its own set of issues:

IOError: [Errno 13] Permission denied: '../../src/hyperv/hyperv_wmi.generated.h' types_typedef = open_and_print(os.path.join(output_dirname, "esx_vi_types.generated.typedef")) File "../../src/esx/esx_vi_generator.py", line 1492, in open_and_print return open(filename, "wb") IOError: [Errno 13] Permission denied:

Sounds like we have a case of a file that gets included in the tarballs, but where the makefile rules think it is out of date and tries to regenerate it. Either it should always be generated (and not part of the tarball), or we need to audit the dependencies to make sure that it does not need rebuilding when used from a pristine tarball. I'll try to take a look into this one later this week (my past experience with dist bugs like this is that they take a good chunk of dedicated time on my part to get to a real root cause).

...

$ git revert 1bfb47dfe61c3cf9a716db072cbe22f26e980081 [master f5a9a90] Revert "Make ESX & Hyper-V code generator safe with parallel builds"

So that means that our conversion to using a stamp file is the real culprit. It may be as simple as just making sure the stamp file is also part of the tarball.

Ahh, right. So the ESX generator explicitly sets the files readonly when it creates them. So when they are in the dist, and the stmp file is not, you hit the EPERM issue. I copied the make rules from the python generator, which also does not include the stamp file in dist. The difference is though that the python generator doesn't make its files readonly Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|