FUD about files over 10MB

[Martin Geisler]

Thomas Arendsen Hein <thomas at intevation.de> writes:

> * Martin Geisler <mg at lazybytes.net> [20100316 11:40]:
>> I just made some quick benchmarking where I added larger and larger
>> files, and using the contrib/memory.py extension, I saw a peak memory
>> usage of 3*N when adding, and 2*N when doing a checkout. I didn't test
>> merging or diffing.
>
> I think committing new versions of a big file might cause extra
> memory usage.

Yes, you're right... scary numbers below. I tested with a 256 MB file
which I made by concatenating a bunch of JPEG images.

Committing the first version required 768 MB.

I then overwrote it with another 256 MB file and now every call to 'hg
status' requires 768 MB(!) and takes some seconds to complete.

The new file has exactly the same size as the old, so we must read it to
see that it differs. I guess we fully reconstruct old file in memory to
compare the two files? They differ at byte 5, so this is quite a waste.

Committing the changed file requires 1509 MB. After the commit, .hg
takes up 510 MB, which is entirely reasonable since the files were very
different and already compressed.

-- 
Martin Geisler

Fast and powerful revision control: http://mercurial.selenic.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100318/9604d2f0/attachment.pgp>