[PATCH RFC/EXAMPLE] tags: have a different cache file per filter level

[Gregory Szorc]

On Sat, Mar 21, 2015 at 6:33 PM, Pierre-Yves David <
pierre-yves.david at ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david at fb.com>
> # Date 1426968613 25200
> #      Sat Mar 21 13:10:13 2015 -0700
> # Node ID e7a02537f7084036e2dec01129e96d736f5ea706
> # Parent  1cfded2fa1a92ee9b55d1f62675569e340a39083
> tags: have a different cache file per filter level
>
> Currently whichever filter level ask for tags last will write the data on
> disk.
> This create massive issue when tags are read for "visible" and "unfiltered"
> on large and multi headed repository (like Mozilla central). See issue4550
> for
> details
>
> We'll want the various cache level to collaborate together (same as want
> branchhead cache are doing). But this is not done in this changesets.
>
>
I strongly support a separate cache per filter. And, I think this patch is
a step in the right direction and would address the immediate concerns in
issue 4550.

However, I have concerns:

1) Each cache will redundantly compute the same .hgtags filenode mappings
for many heads. On large repos (in terms of files and heads), the
performance penalty could be significant.

2) I believe we still have lingering issues with cache invalidation when
some history rewriting operations occur. The bulk of the performance issues
are due to .hgtags filenode resolution. If we could make that mapping less
prone to invalidation, it would eliminate most performance issues.

3) Since clients need to create the tags cache after clone and this could
be expensive, I'd eventually like to "bootstrap" the process via a bundle2
part (containing probably just the .hgtags filenode mapping). This should
be considered for any tags cache refactor.

4) Given 1-3, if we adopt this patch as-is and refactor tags cache later,
we'll have to create a new tags cache file implementation. Part of me
thinks we should do all this at one time so there is 1 transition, not 2.

Between my earlier patch to create a new .hgtags filenode cache and this
patch, I think we're 80% of the way to a solution that combines both forms.
Give me the green light, and I'll hack on it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20150323/01421fa3/attachment.html>