[PATCH 1 of 3 RFC] scmutil: support background file closing

Gregory Szorc gregory.szorc at gmail.com
Fri Jan 8 12:34:08 CST 2016


On Fri, Jan 8, 2016 at 8:49 AM, Bryan O'Sullivan <bos at serpentine.com> wrote:

>
> On Mon, Jan 4, 2016 at 12:12 PM, Gregory Szorc <gregory.szorc at gmail.com>
> wrote:
>
>> Is this more appetizing?
>>
>
> Not really. Here's my worry: most Mercurial developers don't work on
> Windows, and won't have any clue when they should or should not be using
> this background file closer if they're forced to think about it.
>

The criteria for using background file closer are as follows:

1) Are you potentially performing writes/appends to thousands of files?
2) Are the files opened once per your explicitly-defined block (via a
context manager) so there is no potential for a race between closing and
reopening?

The scenarios meeting this criteria today include:

* Changegroup application (assuming each path only exists once in a
changegroup)
* Stream clone application
* Working copy update (and this already has a solution: worker threads -
although it isn't implemented on Windows)

Of these, I only care about stream clones for the moment.

As patch 3 demonstrates, this hack make streaming clones for
mozilla-central on Windows over 4 minutes faster on my machine! Mozilla's
Windows automation engineers are complaining about Mercurial clone
performance to me and short of implementing a new store format that relies
on fewer files (see https://www.mercurial-scm.org/wiki/PackedRepoPlan) or
implementing our own 3rd party clone mechanism [that relies on the same
hacks as this series], I don't see a path forward.

I'll send a RFC V2 that might win you over.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20160108/2a3ca8f2/attachment.html>


More information about the Mercurial-devel mailing list