So I was checking out the repo the other day and it took ages. So it got me thinking what might be the problem. Looks like a part of it is that our pack is split among ~250 files. Therefore when somebody does checkout git needs to repack it into a single pack every time. And this may take ages on such slow processor as Atom is. However, reading some docs on this it looks like 'git gc --aggressive' is not advised rather than 'git repack'.

On Thu, Aug 03, 2017 at 08:57:47AM +0100, Daniel P. Berrange wrote: >On Thu, Aug 03, 2017 at 09:16:13AM +0200, Michal Privoznik wrote: >> So I was checking out the repo the other day and it took ages. So it got >> me thinking what might be the problem. Looks like a part of it is that >> our pack is split among ~250 files. Therefore when somebody does >> checkout git needs to repack it into a single pack every time. And this >> may take ages on such slow processor as Atom is. However, reading some >> docs on this it looks like 'git gc --aggressive' is not advised rather >> than 'git repack'. > >I created a 'tmp' repo and ran 'repack' on it, but afaict, there's no >appreciable difference. > Other thought I had was having the po/*.po files removed from the repo. DV would add them to the release, but they would not take 100MB in the repository when they are not needed for development builds. It would not cut down on the data that need to be transferred since they are in the history, but at least would take some weight off the repo. After few years we could do a split as linux did and have one repo with complete history and one that goes from v4.0.0 onwards.

On 08/03/2017 09:57 AM, Daniel P. Berrange wrote: > On Thu, Aug 03, 2017 at 09:16:13AM +0200, Michal Privoznik wrote: >> So I was checking out the repo the other day and it took ages. So it got >> me thinking what might be the problem. Looks like a part of it is that >> our pack is split among ~250 files. Therefore when somebody does >> checkout git needs to repack it into a single pack every time. And this >> may take ages on such slow processor as Atom is. However, reading some >> docs on this it looks like 'git gc --aggressive' is not advised rather >> than 'git repack'. > > I created a 'tmp' repo and ran 'repack' on it, but afaict, there's no > appreciable difference. In fact, there is. I just finished running 'git repack -a -d' over the 'tmp' repo and here are the results: $ time git clone git://libvirt.org/libvirt.git libvirt_temp.git Cloning into 'libvirt_temp.git'... remote: Counting objects: 236385, done. remote: Compressing objects: 100% (38422/38422), done. remote: Total 236385 (delta 200296), reused 232761 (delta 196975) Receiving objects: 100% (236385/236385), 297.08 MiB | 5.55 MiB/s, done. Resolving deltas: 100% (200296/200296), done. real 2m40.089s user 1m2.831s sys 0m2.970s $ time git clone git://libvirt.org/tmp tmp.git Cloning into 'tmp.git'... remote: Counting objects: 236365, done. remote: Compressing objects: 100% (35400/35400), done. remote: Total 236365 (delta 200277), reused 236065 (delta 199977) Receiving objects: 100% (236365/236365), 297.19 MiB | 6.17 MiB/s, done. Resolving deltas: 100% (200277/200277), done. real 1m16.209s user 1m7.782s sys 0m2.940s In the first case, the network transmission took ~54s, so prep work on server took ~1m45s. In the second case, network transmission took 48s, so prep work took just ~28s. Therefore I think it makes sense to run the command. If nobody objects I can do that later today.

On Thu, Aug 03, 2017 at 09:16:13 +0200, Michal Privoznik wrote: > So I was checking out the repo the other day and it took ages. So it got > me thinking what might be the problem. Looks like a part of it is that > our pack is split among ~250 files. Therefore when somebody does 250 files should not be an issue on today's computers. Maybe 2 orders of magnitude more would pose some issues. > checkout git needs to repack it into a single pack every time. And this > may take ages on such slow processor as Atom is. However, reading some > docs on this it looks like 'git gc --aggressive' is not advised rather > than 'git repack'. > > Any thoughts? Clone from github or some other hosting with more network conectivity/CPU power if you are bothered by this?

-- libvir-list mailing list libvir-list(a)redhat.com https://www.redhat.com/mailman/listinfo/libvir-list