Unifying sparse and narrow "profiles"
Augie Fackler
raf at durin42.com
Wed Mar 29 22:47:44 EDT 2017
(+martinvonz, durham)
> On Mar 29, 2017, at 1:05 PM, Gregory Szorc <gregory.szorc at gmail.com> wrote:
>
> Mozilla has added tens of thousands of files to the Firefox repo in the past few months and we have plans to add tens of thousands more shortly. Working directory update times (especially in automation, which has to do fresh checkouts with a somewhat high frequency since we rely on ephemeral compute instances) were borderline tolerable before. With the addition of tens of thousands of new files, working directory updates are starting to put noticeable strain on systems.
>
> Mozilla can make due with sparse checkouts - we don't yet have so much repo data that we need narrow clone, although narrow would be useful.
>
> Facebook's sparse checkout extension has existed for years and my understanding is it gets the job done. When I asked at the sprint why sparse isn't part of the core distribution, someone mentioned it is because sparse and narrow have different, competing concepts for defining a sparse/narrow profile and these will need to be reconciled before either can be accepted into core.
I think we’ve mostly stopped caring about profiles for our users - Martin, can you confirm that?
> Is there a timeline for unifying the profiles and adding sparse to the core distribution?
This is mostly on me, I think, to clean up narrow and make it so that it can satisfy the varying modes of operation:
0) narrow working directory only
1) narrow working directory and history, but no eliding of irrelevant changes
2) “full narrow”, including elision of irrelevant changes
What we’re using at Google is mode 2, but that’s also the most server-expensive and the most likely to have bugs. It shouldn’t be /too/ much work to reconcile our implementation with sparse, add profiles support, and default to mode 0 or 1.
Durham, do you have opinions on this? Is it a fair assumption on my part that you’d rather we maintain this horror than you?
(Also, are there any docs I should read about your sparse stuff and profiles?)
> Also, if I had to use sparse as it is now, what recommendations would you have? Should I create an /.hgsparse in the repo? Or should I perhaps "hide" the file elsewhere and use `hg sparse --import-rules`? Keep in mind that Mozilla doesn't have as tightly managed setup as other companies do. Once we support sparse in any way, we have to think about BC and what happens when e.g. old versions of Mercurial/sparse are used on a new revision.
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
More information about the Mercurial-devel
mailing list