Experiments with LZMA compression of bundles
Sune Foldager
cryo at cyanite.org
Fri May 20 02:20:33 CDT 2011
Just for fun, I hacked in LZMA compression support in Mercurial's bundle
system, so I could test how much compression that might give us. I
include the results below.
I used pyliblzma, which in turn depends on liblzma2 (used by xz). I used
default settings, which, as I understand, use a rather large amount of
memory, but this can be tuned. Hacking it into Mercurial was quite
simple, although the various systems handling bundles is disseminated
throughout several modules, so I only made it work for on-disk bundles.
-Sune
Results:
The hardware is a 6-core Xeon (X5650) with enough ram and hd.
cryo at firefly:~/test$ time hg bundle --all --type bzip2 bundle.bzip2
10225 changesets found
real 1m57.285s
user 1m53.450s
sys 0m0.840s
cryo at firefly:~/test$ time hg bundle --all --type lzma bundle.lzma
10225 changesets found
real 2m10.215s
user 2m9.110s
sys 0m0.760s
cryo at firefly:~/test$ ll -h bundle*
-rw-r--r-- 1 cryo cryo 31M 2011-05-19 15:57 bundle.bzip2
-rw-r--r-- 1 cryo cryo 18M 2011-05-19 16:01 bundle.lzma
...so from 31M to 18M using 13 more seconds in compression. Not bad. The
repository is normally 87M (using GD), or 276M without GD.
Unbundling is faster with LZMA:
cryo at firefly:~/test2$ time hg unbundle ../bundle.bzip2
adding changesets
adding manifests
adding file changes
added 10225 changesets with 60128 changes to 7764 files (+28 heads)
(run 'hg heads' to see heads)
real 0m58.882s
user 0m57.380s
sys 0m1.230s
cryo at firefly:~/test2$ time hg unbundle ../bundle.lzma
adding changesets
adding manifests
adding file changes
added 10225 changesets with 60128 changes to 7764 files (+28 heads)
(run 'hg heads' to see heads)
real 0m38.786s
user 0m36.930s
sys 0m1.560s
More information about the Mercurial-devel
mailing list