Re: [libvirt] PATCH: 8/11: Make better use of linker scripts

On Thu, Oct 30, 2008 at 01:41:14PM +0000, Daniel P. Berrange wrote: > The libvirt.so file currently whitelists all the symbols in our public > API. libvirtd and virsh, however, also want to use a bunch of our so > called 'private' symbols which aren't in the public API. For saferead > and safewrite() we dealt with this by compiling the code twice with > some nasty macro tricks to give the function a different name to avoid > dup symbols when statically linking. For the other APIs, we prefixed > them with __ and then just added them to the exports. > > Neither option is very good because they both impose a burden on the > source code - needing to append __ everywhere and write crazy macros. > Each time we want to export another symbol, we have to make lots of > manual changes to add __. This was OK if it was just a handful of > symbols, but we're now upto 30 odd, and its not scaling. When we > make drivers modular we'll be adding 100's more symbols. > > As an aside, we use ELF library versioning when linking, via the > libtool -version-info flag. Since we maintain ABI compat going > forwards though, the version in the eventual .so is always going > to be 'libvirt.so.0'. This is sub-optimal because you may build > a binary that requires 'virDomainBlockPeek' which appeared in > libvirt 0.4.2, but the ELF version cannot validate this. There is > nothing stopping your application launching against libvirt 0.3.0 > and then aborting with a linker failure sometime while it is > running when it eventually referneces the non-existant symbol. > Likewise RPM automatic versioning hooks off the ELF version too, > so that's not preventing you installing too old a libvirt for > your app's needs. > > A far better solution to all these problems has been sitting in > front of us the whole time. Namely we need to make full use of > the linkers symbol version scripts. > > Our current libvirt_sym.version script is defining an unversioned > set of symbols. This only enforces what's exported, and can do > nothing about version compatability. So instead we need to switch > to a fully versioned script, where every symbol we export is tagged > with the libvirt version number at which it was introduced. > > Taking the virDomainBlockPeek example again, this appeared in the > libvirt 0.4.2 release, so we'd add a section to the linker script > > LIBVIRT_0.4.2 { > global: > virDomainBlockPeek; > virDomainMemoryPeek; > } LIBVIRT_0.4.1; > > Then 0.4.5 added in storage pool discovery so you get another > section > > LIBVIRT_0.4.5 { > global: > virConnectFindStoragePoolSources; > } LIBVIRT_0.4.2; > > And so on for every release. > > The resulting libvirt.so file will thus gain metadata listing all > the ABI versions for all symbols it includes. > > That deals with public APIs. Now we also want to export some internal > APIs whose semantics/syntax may arbitrarily change between releases. > Thus they should not be included in any of the formal public version > sections. > > Instead they should be in a seperate versioned section, whose version > changes on every release. This will ensure that if libvirtd or virsh > link to some internal symbols, they are guareteened to only run > against the exactly same libvirt.so they were linked to. This will > avoid some nasty bugs our users have hit where they installed a custom > libvirt version on their OS which already had another version installed, > and then libvirtd crashed/behave wierdly/etc > > To do this we rename libvirt_sym.version to libvirt_sym.version.in > and add a section called > > LIBVIRT_PRIVATE_@VERSION@ { > > global: > /* libvirt_internal.h */ > debugFlag; > virStateInitialize; > virStateCleanup; > virStateReload; > virStateActive; > > .... more private symbols... > } > > The @VERSION@ gets subsituted by the version number of the libvirt > release by configure. > > Do a build with this all active and look at the resulting libvirt.so > using objdump (or eu-readelf). > > # objdump -p src/.libs/libvirt.so > Version definitions: > 1 0x01 0x0e5a1d10 libvirt.so.0 > 2 0x00 0x0af6bd33 LIBVIRT_0.0.3 > 3 0x00 0x0af6bd35 LIBVIRT_0.0.5 > LIBVIRT_0.0.3 > 4 0x00 0x0af6be30 LIBVIRT_0.1.0 > LIBVIRT_0.0.5 > 5 0x00 0x0af6be31 LIBVIRT_0.1.1 > LIBVIRT_0.1.0 > 6 0x00 0x0af6be34 LIBVIRT_0.1.4 > LIBVIRT_0.1.1 > 7 0x00 0x0af6be35 LIBVIRT_0.1.5 > LIBVIRT_0.1.4 > 8 0x00 0x0af6be39 LIBVIRT_0.1.9 > LIBVIRT_0.1.5 > 9 0x00 0x0af6bb30 LIBVIRT_0.2.0 > LIBVIRT_0.1.9 > 10 0x00 0x0af6bb31 LIBVIRT_0.2.1 > LIBVIRT_0.2.0 > 11 0x00 0x0af6bb33 LIBVIRT_0.2.3 > LIBVIRT_0.2.1 > 12 0x00 0x0af6bc30 LIBVIRT_0.3.0 > LIBVIRT_0.2.3 > 13 0x00 0x0af6bc32 LIBVIRT_0.3.2 > LIBVIRT_0.3.0 > 14 0x00 0x0af6bc33 LIBVIRT_0.3.3 > LIBVIRT_0.3.2 > 15 0x00 0x0af6b930 LIBVIRT_0.4.0 > LIBVIRT_0.3.3 > 16 0x00 0x0af6b931 LIBVIRT_0.4.1 > LIBVIRT_0.4.0 > 17 0x00 0x0af6b932 LIBVIRT_0.4.2 > LIBVIRT_0.4.1 > 18 0x00 0x0af6b935 LIBVIRT_0.4.5 > LIBVIRT_0.4.2 > 19 0x00 0x0af6ba30 LIBVIRT_0.5.0 > LIBVIRT_0.4.5 > 20 0x00 0x09e39b06 LIBVIRT_PRIVATE_0.4.6 > > > You can see that as well as the main SONAME libvirt.so.0, we've > got version info for each release which introduced new public > API symbols, as well as our versioned private symbols. If you > look at glibc's libc.so, you'll see a similar set of versioned > interfaces. > > Then take a look at the virsh binary, and see that it has references > to all of the versions corresponding to symbols it uses. You can > also see that it used some libvirt internal symbols, by presence of > the LIBVIRT_PRIVATE_0.4.6 reference. This ensures that this virsh > binary can't accidentally be used against libvirt 0.4.5 or 0.4.7 > where it'd likely crash at runtime > > # objdump -p src/.libs/virsh > Version References: > required from libpthread.so.0: > 0x09691a75 0x00 10 GLIBC_2.2.5 > required from libc.so.6: > 0x0d696918 0x00 19 GLIBC_2.8 > 0x09691974 0x00 13 GLIBC_2.3.4 > 0x09691a75 0x00 04 GLIBC_2.2.5 > required from libvirt.so.0: > 0x0af6ba30 0x00 22 LIBVIRT_0.5.0 > 0x0af6bc30 0x00 21 LIBVIRT_0.3.0 > 0x0af6bc33 0x00 20 LIBVIRT_0.3.3 > 0x0af6be34 0x00 18 LIBVIRT_0.1.4 > 0x0af6be35 0x00 17 LIBVIRT_0.1.5 > 0x0af6b935 0x00 16 LIBVIRT_0.4.5 > 0x0af6be31 0x00 15 LIBVIRT_0.1.1 > 0x0af6bc32 0x00 14 LIBVIRT_0.3.2 > 0x0af6be39 0x00 12 LIBVIRT_0.1.9 > 0x0af6bd33 0x00 11 LIBVIRT_0.0.3 > 0x0af6bb31 0x00 09 LIBVIRT_0.2.1 > 0x09e39b06 0x00 08 LIBVIRT_PRIVATE_0.4.6 > 0x0af6b930 0x00 07 LIBVIRT_0.4.0 > 0x0af6bb30 0x00 06 LIBVIRT_0.2.0 > 0x0af6be30 0x00 05 LIBVIRT_0.1.0 > 0x0af6bb33 0x00 03 LIBVIRT_0.2.3 > 0x0af6b931 0x00 02 LIBVIRT_0.4.1 As a further example, of something using only a subset of the API, consider a really trivial 3 liner demo program which uses 3 functions: $ cat demo.c #include <stdio.h> #include <libvirt/libvirt.h> int main() { char buf[10]; virDomainPtr dom; virConnectPtr cb = virConnectOpen(NULL); dom = virDomainLookupByName(cb, "foo"); virDomainBlockPeek(dom, "hda", 0, 10, buf, 0); } $ gcc -o demo -lvirt demo.c $ objdump -p demo Version References: required from libc.so.6: 0x09691a75 0x00 03 GLIBC_2.2.5 required from libvirt.so.0: 0x0af6b932 0x00 04 LIBVIRT_0.4.2 0x0af6bd33 0x00 02 LIBVIRT_0.0.3 So you can see the linker only encodes the exact versions used by the 3 vir* methods it calls. 0.0.3 for virConnectOpen and virDomainLookupByName, and 0.4.2 for virDomainBlockPeek.