Importing to hg from perforce with large binary files

[Mark Mason]

Hello,

I'm trying to convert a p4 depot to Mercurial. The depot in question
contains many large binary files, which I would like to handle under
Mercurial with one of the big/binary file extensions.

So, I downloaded and enabled both the perfarce extension and the bigfiles
extension, crossed my fingers and hoped that magically all the large files
detected during the import would be automatically shuffled off to the large
binaries directory, and ran something like:

    hg clone p4://p4server-hostname.domain.com:1666/mason-test mason-test

After several hours of crunching through the data, I ended up with a 25G hg
repo, and an empty binaries directory :(. I was hopeful that this would just
work[tm], but not too surprised that it didn't. I'm a reasonably fluent
python programmer, but I'm not familiar enough with the inner workings of
Mercurial yet to craft my own extension to do this.

How have people handled this in the past? Is there a combination of
extensions that will work, or will I have to do the conversion manually?
This is part of us evaluating Mercurial as our new SCM system here at work,
and importing from our current systems (p4 and CVS mainly) is very high on
our must-have list. I'm the one leading the Mercurial evaluation effort, and
I'd really like to see it succeed.

Or do I have to frankenstein the binary file support into the perfarce
extension? I'm not sure I have the time to do that within our evaluation
schedule (besides, I also have my day job of writing firmware), even though
it sounds like fun.

Unfortunately, simply excluding the binaries and putting them somewhere else
outside of SCM isn't an option in this case.

On a related note - that 25G repo takes just over a hour to clone through
Mercurial server. A tar transfer of the repo from behind the Mercurial
server's back takes only 13 minutes (so it's not a disk speed issue), and a
p4 sync -f ... of the original p4 depot that I converted from takes 7
minutes (of course, it's only checking out the 7G that makes up the tip of
branches imported). Any ideas why it's taking so long to do the clone?
Performance is also a significant consideration for us.

Thanks very much in advance for any advice, pointers, etc. I really want
Mercurial to succeed in this evaluation, so any help is very much
appreciated.

Cheers,
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial/attachments/20110126/cdb0bfa7/attachment.htm>