Experiments with LZMA compression of bundles

Sune Foldager cryo at cyanite.org
Fri May 20 02:20:33 CDT 2011


Just for fun, I hacked in LZMA compression support in Mercurial's bundle 
system, so I could test how much compression that might give us. I 
include the results below.

I used pyliblzma, which in turn depends on liblzma2 (used by xz). I used 
default settings, which, as I understand, use a rather large amount of 
memory, but this can be tuned. Hacking it into Mercurial was quite 
simple, although the various systems handling bundles is disseminated 
throughout several modules, so I only made it work for on-disk bundles.

-Sune

Results:
The hardware is a 6-core Xeon (X5650) with enough ram and hd.

cryo at firefly:~/test$ time hg bundle --all --type bzip2 bundle.bzip2
10225 changesets found

real 1m57.285s
user 1m53.450s
sys  0m0.840s

cryo at firefly:~/test$ time hg bundle --all --type lzma bundle.lzma
10225 changesets found

real 2m10.215s
user 2m9.110s
sys  0m0.760s

cryo at firefly:~/test$ ll -h bundle*
-rw-r--r-- 1 cryo cryo 31M 2011-05-19 15:57 bundle.bzip2
-rw-r--r-- 1 cryo cryo 18M 2011-05-19 16:01 bundle.lzma

...so from 31M to 18M using 13 more seconds in compression. Not bad. The 
repository is normally 87M (using GD), or 276M without GD.

Unbundling is faster with LZMA:

cryo at firefly:~/test2$ time hg unbundle ../bundle.bzip2
adding changesets
adding manifests
adding file changes
added 10225 changesets with 60128 changes to 7764 files (+28 heads)
(run 'hg heads' to see heads)

real 0m58.882s
user 0m57.380s
sys  0m1.230s

cryo at firefly:~/test2$ time hg unbundle ../bundle.lzma
adding changesets
adding manifests
adding file changes
added 10225 changesets with 60128 changes to 7764 files (+28 heads)
(run 'hg heads' to see heads)

real 0m38.786s
user 0m36.930s
sys  0m1.560s


More information about the Mercurial-devel mailing list