[PATCH 01 of 13 sparse] sparse: vendor Facebook-developed extension

Durham Goode durham at fb.com
Mon Jul 3 18:51:23 EDT 2017



On 7/1/17 7:14 PM, Gregory Szorc wrote:
> On Sat, Jul 1, 2017 at 6:55 PM, Gregory Szorc <gregory.szorc at gmail.com
> <mailto:gregory.szorc at gmail.com>> wrote:
>
>     # HG changeset patch
>     # User Gregory Szorc <gregory.szorc at gmail.com
>     <mailto:gregory.szorc at gmail.com>>
>     # Date 1498931009 25200
>     #      Sat Jul 01 10:43:29 2017 -0700
>     # Node ID 0c01dd19a7e2e57dd9d1abba3cb9643f37947c8c
>     # Parent  4d780d510b44ad2ae3bb39f3816c95598704d1eb
>     sparse: vendor Facebook-developed extension
>
>
> I found out this past week that there is a desire to add ~100,000 files
> to the Firefox repo in the next few months - before Mercurial's 4.4
> release. That would bring us over 300,000 files in our working
> directory. We're already experiencing some working directory scaling
> pain (especially on Windows) and I fear that moving forward with the
> planned file count increases will push us past a poor performance
> tipping point. So we're likely looking at leveraging sparse in one form
> or another in the next few months.
>
> It is easier for Mozilla to support sparse checkout if it is baked into
> the official distribution - even if there are no BC guarantees.
>
> So my goal with this series is to get a functional sparse checkout -
> however hacky - into the core distribution for 4.3 so that Mozilla can
> leverage it in some capacity (even if we're writing sparse profile files
> manually and using a minimal custom extension to trigger sparseness). I
> would also like to move as much of the extension into core as possible
> so that narrow developers and others can start actively seeing and
> improving the code. I have a feeling there are a number of performance
> wins that we can realize by having tighter integration between sparse
> filtering and working directory operations.
>
> As I move code from the extension to core, I'm attempting to make as
> many things as possible no-op unless sparseness is enabled. This
> *should* mean that narrow isn't affected by the presence of sparse code
> in core. I'm also trying to not break Facebook with changes: if they
> need to e.g. rename their existing extension to "fbsparse," it should
> work, even with the presence of sparse code in core.

The series looks reasonable to me (assuming people are ok with the 
initial vendoring), and I'm all for getting sparse into core, even if 
it's initially experimental.

+1 on moving things to a special sparse module instead of merge.py too.


More information about the Mercurial-devel mailing list