problem cloning large file (32MB) on Windows/Network Shares

[Matt Mackall]

On Mon, 2011-01-17 at 23:37 +0200, Fred Sundvik wrote:
> Ok, I'm sorry, I didn't think about the case of merging files. And yes I 
> understand you, I actually know a bit about LCS algorithms, as I was 
> evaluating a few different ones and ended implementing one in the 
> context of delta compression of arrays for network traffic.  Anyway 
> there should be plenty of memory in modern computers, and as every 
> command runs as a separate process, there shouldn't be any fragmentation 
> to talk of, so support for quite big files without running out of memory 
> should be possible.

Putting aside silly operating system bugs, 32MB files are just not an
issue for us.

But unfortunately, lots of machines still run 32-bit code in a 2GB
address space (even though the machine itself may have lots more RAM).
Trying to do delta compression/decompression on a 500MB file in a 2G
address space is like trying to assemble a desk in a closet. 
You might be able to do it by tediously shuffling parts to and from your
garage, but it'd make a lot more sense to just build it in your 64-bit
living room.

And once you move beyond 1G, you're now in a range where it's simply
impractical to push many copies of file history around on today's
typical networks. Mercurial currently handles files up to 2G on 64-bit
machines, and I can count the number of people who've complained about
this limit on one hand.

-- 
Mathematics is the supreme nostalgia of our time.