Re: [libvirt] PATCH: 8/11: Make better use of linker scripts

The libvirt.so file currently whitelists all the symbols in our public API. libvirtd and virsh, however, also want to use a bunch of our so called 'private' symbols which aren't in the public API. For saferead and safewrite() we dealt with this by compiling the code twice with some nasty macro tricks to give the function a different name to avoid dup symbols when statically linking. For the other APIs, we prefixed them with __ and then just added them to the exports. Neither option is very good because they both impose a burden on the source code - needing to append __ everywhere and write crazy macros. Each time we want to export another symbol, we have to make lots of manual changes to add __. This was OK if it was just a handful of symbols, but we're now upto 30 odd, and its not scaling. When we make drivers modular we'll be adding 100's more symbols. As an aside, we use ELF library versioning when linking, via the libtool -version-info flag. Since we maintain ABI compat going forwards though, the version in the eventual .so is always going to be 'libvirt.so.0'. This is sub-optimal because you may build a binary that requires 'virDomainBlockPeek' which appeared in libvirt 0.4.2, but the ELF version cannot validate this. There is nothing stopping your application launching against libvirt 0.3.0 and then aborting with a linker failure sometime while it is running when it eventually referneces the non-existant symbol. Likewise RPM automatic versioning hooks off the ELF version too, so that's not preventing you installing too old a libvirt for your app's needs. A far better solution to all these problems has been sitting in front of us the whole time. Namely we need to make full use of the linkers symbol version scripts. Our current libvirt_sym.version script is defining an unversioned set of symbols. This only enforces what's exported, and can do nothing about version compatability. So instead we need to switch to a fully versioned script, where every symbol we export is tagged with the libvirt version number at which it was introduced. Taking the virDomainBlockPeek example again, this appeared in the libvirt 0.4.2 release, so we'd add a section to the linker script LIBVIRT_0.4.2 { global: virDomainBlockPeek; virDomainMemoryPeek; } LIBVIRT_0.4.1; Then 0.4.5 added in storage pool discovery so you get another section LIBVIRT_0.4.5 { global: virConnectFindStoragePoolSources; } LIBVIRT_0.4.2; And so on for every release. The resulting libvirt.so file will thus gain metadata listing all the ABI versions for all symbols it includes. That deals with public APIs. Now we also want to export some internal APIs whose semantics/syntax may arbitrarily change between releases. Thus they should not be included in any of the formal public version sections. Instead they should be in a seperate versioned section, whose version changes on every release. This will ensure that if libvirtd or virsh link to some internal symbols, they are guareteened to only run against the exactly same libvirt.so they were linked to. This will avoid some nasty bugs our users have hit where they installed a custom libvirt version on their OS which already had another version installed, and then libvirtd crashed/behave wierdly/etc To do this we rename libvirt_sym.version to libvirt_sym.version.in and add a section called LIBVIRT_PRIVATE_@VERSION@ { global: /* libvirt_internal.h */ debugFlag; virStateInitialize; virStateCleanup; virStateReload; virStateActive; .... more private symbols... } The @VERSION@ gets subsituted by the version number of the libvirt release by configure. Do a build with this all active and look at the resulting libvirt.so using objdump (or eu-readelf). # objdump -p src/.libs/libvirt.so Version definitions: 1 0x01 0x0e5a1d10 libvirt.so.0 2 0x00 0x0af6bd33 LIBVIRT_0.0.3 3 0x00 0x0af6bd35 LIBVIRT_0.0.5 LIBVIRT_0.0.3 4 0x00 0x0af6be30 LIBVIRT_0.1.0 LIBVIRT_0.0.5 5 0x00 0x0af6be31 LIBVIRT_0.1.1 LIBVIRT_0.1.0 6 0x00 0x0af6be34 LIBVIRT_0.1.4 LIBVIRT_0.1.1 7 0x00 0x0af6be35 LIBVIRT_0.1.5 LIBVIRT_0.1.4 8 0x00 0x0af6be39 LIBVIRT_0.1.9 LIBVIRT_0.1.5 9 0x00 0x0af6bb30 LIBVIRT_0.2.0 LIBVIRT_0.1.9 10 0x00 0x0af6bb31 LIBVIRT_0.2.1 LIBVIRT_0.2.0 11 0x00 0x0af6bb33 LIBVIRT_0.2.3 LIBVIRT_0.2.1 12 0x00 0x0af6bc30 LIBVIRT_0.3.0 LIBVIRT_0.2.3 13 0x00 0x0af6bc32 LIBVIRT_0.3.2 LIBVIRT_0.3.0 14 0x00 0x0af6bc33 LIBVIRT_0.3.3 LIBVIRT_0.3.2 15 0x00 0x0af6b930 LIBVIRT_0.4.0 LIBVIRT_0.3.3 16 0x00 0x0af6b931 LIBVIRT_0.4.1 LIBVIRT_0.4.0 17 0x00 0x0af6b932 LIBVIRT_0.4.2 LIBVIRT_0.4.1 18 0x00 0x0af6b935 LIBVIRT_0.4.5 LIBVIRT_0.4.2 19 0x00 0x0af6ba30 LIBVIRT_0.5.0 LIBVIRT_0.4.5 20 0x00 0x09e39b06 LIBVIRT_PRIVATE_0.4.6 You can see that as well as the main SONAME libvirt.so.0, we've got version info for each release which introduced new public API symbols, as well as our versioned private symbols. If you look at glibc's libc.so, you'll see a similar set of versioned interfaces. Then take a look at the virsh binary, and see that it has references to all of the versions corresponding to symbols it uses. You can also see that it used some libvirt internal symbols, by presence of the LIBVIRT_PRIVATE_0.4.6 reference. This ensures that this virsh binary can't accidentally be used against libvirt 0.4.5 or 0.4.7 where it'd likely crash at runtime # objdump -p src/.libs/virsh Version References: required from libpthread.so.0: 0x09691a75 0x00 10 GLIBC_2.2.5 required from libc.so.6: 0x0d696918 0x00 19 GLIBC_2.8 0x09691974 0x00 13 GLIBC_2.3.4 0x09691a75 0x00 04 GLIBC_2.2.5 required from libvirt.so.0: 0x0af6ba30 0x00 22 LIBVIRT_0.5.0 0x0af6bc30 0x00 21 LIBVIRT_0.3.0 0x0af6bc33 0x00 20 LIBVIRT_0.3.3 0x0af6be34 0x00 18 LIBVIRT_0.1.4 0x0af6be35 0x00 17 LIBVIRT_0.1.5 0x0af6b935 0x00 16 LIBVIRT_0.4.5 0x0af6be31 0x00 15 LIBVIRT_0.1.1 0x0af6bc32 0x00 14 LIBVIRT_0.3.2 0x0af6be39 0x00 12 LIBVIRT_0.1.9 0x0af6bd33 0x00 11 LIBVIRT_0.0.3 0x0af6bb31 0x00 09 LIBVIRT_0.2.1 0x09e39b06 0x00 08 LIBVIRT_PRIVATE_0.4.6 0x0af6b930 0x00 07 LIBVIRT_0.4.0 0x0af6bb30 0x00 06 LIBVIRT_0.2.0 0x0af6be30 0x00 05 LIBVIRT_0.1.0 0x0af6bb33 0x00 03 LIBVIRT_0.2.3 0x0af6b931 0x00 02 LIBVIRT_0.4.1