Re: [libvirt] RFC: Splitting python binding out into a separate repo & ading to PyPi

As everyone knows, we have historically always shipped the python binding as part of the libvirt primary tar.gz distribution. In some ways that has simplified life for people, since we know they'll always have a libvirt python that matches their libvirt C library. At the same time though, this policy of ours is causing increasing amounts of pain for a number of our downstream users. In OpenStack, in particular, their development and test environments aim to be able to not rely on any system installed python packages. They use a virtualenv and pip to install all python deps from PyPi (equivalent of Perl's CPAN in Python world). This approach works for everything except the libvirt Python code which is not available standalone on PyPi. This is causing so much pain that people have suggested taking the libvirt python code we ship and just uploading it to PyPi themselves[1]. This would obviously be a somewhat hostile action to take, but the way we distribute libvirt python is forcing OpenStack to consider such things. In RHEL world too, bundling of libvirt + its python binding is causing pain with the fairly recent concept of "software collections"[2]. This allows users to install multiple versions of languages like Python, Perl, etc on the same box in parallel. To use libvirt python with thse alternate python installs though, requires that they recompile the entire libvirt distribution just to get the Python binding. This is obviously not an approach that works for most people, particularly if they're looking to populate their software collection using 'pip' rather than RPM. Looking on google there are a number of other people asking for libvirt python as a separate module, eg on Stackoverflow[3]. I don't think these issues are going to go away, in fact I think they will likely become more pressing, until the point where some 3rd party takes the step of providing libvirt python bindings themselves. I don't think we want to let ourselves drift into the situation where we loose control over releasing libvirt python bindings. IMHO we should / must listen to our users here before it is too late. We can still release libvirt python at the same time as normal libvirt releases, and require that people update the bindings whenever adding new APIs (if the generator doesn't cope with them). We should simply distribute python as a separate tar.gz, as we do for all other languages, and upload it to PyPi, as well as libvirt.org FTP when doing a release. Obviously there will be some work to separate things out, but I don't see that being insurmountable, since all other language bindings manage to be separate, even when doing code generation. We'd also want to change to use distutils, rather than autoconf, since that's what the python world wants.