Re: [libvirt] [PATCH 0/5] Improve translation handling

This series aims to improve the way we handle translations. The key problems with our current approach - The pot & po files stored in GIT contain huge set of annotations about source file names & line numbers. These are out of date as soon as a change is commited to git following a translation refresh. This makes diffs impossible to meaningfully review, as they are 98% noise, 2% signal. - The po file messages are sorted by source location, so when we move code between files, or rename files, the po file message order changes for no good reason. This makes diffs even more impossible to review. - The po files contain entries for all messages even if most have no translation, bloating size of po/ data stored in git - Whenever 'make dist' is run, it alters all the pot and po files, so developers need to then reset their content to match git HEAD manually. This is caused by having auto-generated content (source file locations) mixed in with the static content (the actual translated strings) Overall the *.po files we store consume 100MB, and when I refresh content from Zanata, it results in git commits that are many 10's of MB in size and impossible todo any meaningful review on. After this series, we only minimized po files in git, with the redundated & outdated source locations info stripped. This stripped info is re-added automatically during build to create the real .po files, that we distribute, and/or upload to translators in Zanata. As a result the po directory is only 19MB in size, and when refreshing from Zanata, we have git commits that clearly show *only* the altered translations, nothing else. The importance of this cannot be overstated - by having these clear diffs I discovered a serious bug in the Zanata client that has been screwing up translations in every project that uses Zanata by adding bogus "fuzzy" annotations. Note that what I've posted here is a cut-down version of what will actually be pushed. I have cut out the 99 other patches that actally change the .po files. There's no meaningful way to review those commits because of the noise in the original .po files we're replacing. I have gone through manual steps, however, to validate that the .po files we generate from the new .mini.po files are identical to the original .po files we used to store, aside from the bogus fuzzy annotations due to broken Zanata. The diffstat shown below covers the full set of patches on the branch at: https://github.com/berrange/libvirt/commits/i18n-4 The killer statistic is: 418787 insertions(+), 4388281 deletions(-) Daniel P. Berrangé (5): po: provide custom make rules for po file management po: remove language list from zanata configuration po: add rules for integration with zanata po: stop storing libvirt.pot in git po: minimize & canonicalize translations stored in git .gitignore | 9 +- ABOUT-NLS | 1 + autogen.sh | 3 +- bootstrap.conf | 38 - build-aux/minimize-po.pl | 37 + configure.ac | 33 +- m4/virt-nls.m4 | 70 + po/Makefile.am | 107 + po/{POTFILES.in => POTFILES} | 0 po/README.md | 75 + po/af.mini.po | 20 + po/af.po | 46395 ------------------------------ po/am.mini.po | 20 + po/am.po | 46395 ------------------------------ po/anp.mini.po | 20 + po/anp.po | 46395 ------------------------------ po/ar.mini.po | 762 + po/ar.po | 46685 ------------------------------ po/{as.po => as.mini.po} | 54246 ++++++++++------------------------- po/ast.mini.po | 20 + po/ast.po | 46395 ------------------------------ po/bal.mini.po | 20 + po/bal.po | 46395 ------------------------------ po/be.mini.po | 21 + po/be.po | 46396 ------------------------------ po/bg.mini.po | 1204 + po/bg.po | 46733 ------------------------------ po/bn.mini.po | 761 + po/bn.po | 46684 ------------------------------ po/bn_IN.mini.po | 9131 ++++++ po/bn_IN.po | 46729 ------------------------------ po/bo.mini.po | 20 + po/bo.po | 46395 ------------------------------ po/br.mini.po | 20 + po/br.po | 46395 ------------------------------ po/brx.mini.po | 20 + po/brx.po | 46395 ------------------------------ po/bs.mini.po | 767 + po/bs.po | 46693 ------------------------------ po/ca.mini.po | 1618 ++ po/ca.po | 46555 ------------------------------ po/cs.mini.po | 4146 +++ po/cs.po | 46751 ------------------------------ po/cy.mini.po | 762 + po/cy.po | 46701 ------------------------------ po/da.mini.po | 982 + po/da.po | 46704 ------------------------------ po/de.mini.po | 22654 +++++++++++++++ po/de.po | 47656 ------------------------------- po/de_CH.mini.po | 20 + po/de_CH.po | 46395 ------------------------------ po/el.mini.po | 724 + po/el.po | 46857 ------------------------------ po/en_GB.mini.po | 22271 +++++++++++++++ po/en_GB.po | 47573 ------------------------------- po/eo.mini.po | 20 + po/eo.po | 46395 ------------------------------ po/es.mini.po | 22622 +++++++++++++++ po/es.po | 47748 ------------------------------- po/et.mini.po | 761 + po/et.po | 46684 ------------------------------ po/eu.mini.po | 761 + po/eu.po | 46684 ------------------------------ po/fa.mini.po | 20 + po/fa.po | 46395 ------------------------------ po/fi.mini.po | 1281 + po/fi.po | 46744 ------------------------------ po/fil.mini.po | 20 + po/fr.mini.po | 4258 +++ po/fr.po | 46907 ------------------------------ po/fur.mini.po | 20 + po/ga.mini.po | 21 + po/gl.mini.po | 761 + po/gl.po | 46700 ------------------------------ po/{gu.po => gu.mini.po} | 54191 ++++++++++------------------------- po/he.mini.po | 761 + po/he.po | 46684 ------------------------------ po/hi.mini.po | 13772 +++++++++ po/hi.po | 47013 ------------------------------- po/hr.mini.po | 21 + po/hr.po | 46396 ------------------------------ po/hu.mini.po | 987 + po/hu.po | 46709 ------------------------------ po/ia.mini.po | 20 + po/ia.po | 46395 ------------------------------ po/id.mini.po | 728 + po/id.po | 46691 ------------------------------ po/ilo.mini.po | 20 + po/ilo.po | 46395 ------------------------------ po/is.mini.po | 761 + po/is.po | 46684 ------------------------------ po/it.mini.po | 6588 +++++ po/it.po | 46663 ------------------------------ po/{ja.po => ja.mini.po} | 54737 ++++++++++------------------------- po/ka.mini.po | 761 + po/ka.po | 46684 ------------------------------ po/kk.mini.po | 20 + po/kk.po | 46395 ------------------------------ po/km.mini.po | 20 + po/km.po | 46395 ------------------------------ po/{kn.po => kn.mini.po} | 54156 ++++++++++------------------------- po/ko.mini.po | 12838 +++++++++ po/ko.po | 46723 ------------------------------ po/kw.mini.po | 21 + po/kw.po | 46396 ------------------------------ po/kw(a)kkcor.mini.po | 21 + po/kw(a)kkcor.po | 46396 ------------------------------ po/kw(a)uccor.mini.po | 21 + po/kw(a)uccor.po | 46396 ------------------------------ po/kw_GB.mini.po | 21 + po/kw_GB.po | 46396 ------------------------------ po/ky.mini.po | 20 + po/ky.po | 46395 ------------------------------ po/libvirt.pot | 46394 ------------------------------ po/lt.mini.po | 762 + po/lt.po | 46685 ------------------------------ po/lv.mini.po | 762 + po/lv.po | 46685 ------------------------------ po/mai.mini.po | 20 + po/mai.po | 46395 ------------------------------ po/mk.mini.po | 1138 + po/mk.po | 46724 ------------------------------ po/{ml.po => ml.mini.po} | 53963 ++++++++++------------------------- po/mn.mini.po | 20 + po/mn.po | 46395 ------------------------------ po/{mr.po => mr.mini.po} | 55484 +++++++++++------------------------- po/ms.mini.po | 189 + po/ms.po | 46580 ------------------------------ po/my.mini.po | 20 + po/nb.mini.po | 492 + po/nb.po | 46667 ------------------------------ po/nds.mini.po | 20 + po/nds.po | 46395 ------------------------------ po/ne.mini.po | 20 + po/ne.po | 46395 ------------------------------ po/nl.mini.po | 9868 +++++++ po/nl.po | 47062 ------------------------------- po/nn.mini.po | 761 + po/nn.po | 46684 ------------------------------ po/nso.mini.po | 761 + po/nso.po | 46684 ------------------------------ po/{or.po => or.mini.po} | 53161 +++++++++------------------------- po/{pa.po => pa.mini.po} | 53711 ++++++++++------------------------- po/pl.mini.po | 10274 +++++++ po/pl.po | 47166 ------------------------------- po/pt.mini.po | 1243 + po/pt.po | 46587 ------------------------------ po/pt_BR.mini.po | 22821 +++++++++++++++ po/pt_BR.po | 47565 ------------------------------- po/ro.mini.po | 762 + po/ro.po | 46685 ------------------------------ po/{ru.po => ru.mini.po} | 54065 ++++++++++------------------------- po/si.mini.po | 761 + po/si.po | 46684 ------------------------------ po/sk.mini.po | 761 + po/sk.po | 46684 ------------------------------ po/sl.mini.po | 762 + po/sl.po | 46685 ------------------------------ po/sq.mini.po | 24 + po/sq.po | 46399 ------------------------------ po/sr.mini.po | 2203 ++ po/sr.po | 46910 ------------------------------ po/sr(a)latin.mini.po | 2204 ++ po/sr(a)latin.po | 46911 ------------------------------ po/sv.mini.po | 2559 ++ po/sv.po | 46696 ------------------------------ po/{ta.po => ta.mini.po} | 54968 +++++++++++------------------------- po/{te.po => te.mini.po} | 53427 ++++++++++------------------------- po/tg.mini.po | 20 + po/tg.po | 46395 ------------------------------ po/th.mini.po | 761 + po/th.po | 46684 ------------------------------ po/tr.mini.po | 761 + po/tr.po | 46684 ------------------------------ po/tw.mini.po | 20 + po/tw.po | 46395 ------------------------------ po/{uk.po => uk.mini.po} | 62575 +++++++++++++++++------------------------ po/ur.mini.po | 761 + po/ur.po | 46684 ------------------------------ po/vi.mini.po | 8674 ++++++ po/vi.po | 47171 ------------------------------- po/wba.mini.po | 20 + po/wba.po | 46395 ------------------------------ po/yo.mini.po | 20 + po/yo.po | 46395 ------------------------------ po/zanata.xml | 99 - po/zh_CN.mini.po | 23613 ++++++++++++++++ po/zh_CN.po | 46701 ------------------------------ po/zh_HK.mini.po | 20 + po/zh_HK.po | 46395 ------------------------------ po/zh_TW.mini.po | 958 + po/zh_TW.po | 46725 ------------------------------ po/zu.mini.po | 761 + po/zu.po | 46684 ------------------------------ src/internal.h | 16 +- src/libvirt-admin.c | 2 + src/libvirt.c | 2 + src/util/virgettext.c | 3 +- tools/virt-host-validate.c | 4 +- 199 files changed, 418787 insertions(+), 4388281 deletions(-) -- 2.14.3