FUD about files over 10MB

[Martin Geisler]

Hi guys,

I would like to make the warning we give when adding large files more
precise. Right now you're told:

  large.jpg: files over 10MB may cause memory and performance problems

This makes people uneasy and leads to questions like this, where a guy
is looking for a way to split his large file into 10 MB pieces "so that
they can be be accepted by my Mercurial repository."

  http://serverfault.com/questions/122737/in-bash-are-wildcard-expansions-guaranteed-to-be-in-order

I just made some quick benchmarking where I added larger and larger
files, and using the contrib/memory.py extension, I saw a peak memory
usage of 3*N when adding, and 2*N when doing a checkout. I didn't test
merging or diffing.

Hence, I would like to tell the user *that* instead of only hinting at
some unspecified problems. Something like

  large.jpg: you will now need at least 30 MB of RAM to work with this repository

Perhaps it would be good to explictly mention that the file is large,
otherwise the warning may seem to come out of the blue? I just don't
know how to formulate this in a brief way.

My current patch:

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1091,10 +1091,10 @@
                     rejected.append(f)
                     continue
                 if st.st_size > 10000000:
-                    self.ui.warn(_("%s: files over 10MB may cause memory and"
-                                   " performance problems\n"
+                    self.ui.warn(_("%s: you will now need at least %d MB "
+                                   "of RAM to work with this repository\n"
                                    "(use 'hg revert %s' to unadd the file)\n")
-                                   % (f, f))
+                                   % (f, 3 * st.st_size // 1000000, f))
                 if not (stat.S_ISREG(st.st_mode) or stat.S_ISLNK(st.st_mode)):
                     self.ui.warn(_("%s not added: only files and symlinks "
                                    "supported currently\n") % f)


I've attached the script I used to combine a bunch of JPG files into
larger files, and the script I used to test with.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: combine.py
Type: text/x-python
Size: 555 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100316/3ef0135d/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.sh
Type: text/x-sh
Size: 250 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100316/3ef0135d/attachment.sh>
-------------- next part --------------

-- 
Martin Geisler

Fast and powerful revision control: http://mercurial.selenic.com/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100316/3ef0135d/attachment.pgp>