[PATCH 6 of 6 V2] obscache: use the obscache to compute the obsolete set

Jun Wu quark at fb.com
Sat May 20 13:43:38 EDT 2017


Excerpts from Jun Wu's message of 2017-05-20 10:37:23 -0700:
> While this does speed up commands like "hg id", people more frequently use
> "hg log [-G]" which will call "ctx.troubles" by default and that requires a
> full "obsolete()" revset in multiple places (unstable, bumped and divergent
> calculation), which won't be helped by this cache.

I take the above back. This is still helpful as it improves the constant of
time complexity. But the below text is still true - the time complexity is
unchanged and a complex caching layer got added. I think we'll have similar
perf win in a cleaner way (no cache invalidation trouble) by writing C code.

> Do you plan to build similar bitmap cache for unstable, bumped, divergent
> too?
> 
> Besides, even with this series, _computeobsoleteset is still O(N). The time
> complexity remains unchanged. This is not a long-term solution.
> 
> I believe the long term solution is to make testing O(1) and remove all
> enumeration for all revsets.
> 
> Excerpts from Pierre-Yves David's message of 2017-05-20 17:30:20 +0200:
> > # HG changeset patch
> > # User Pierre-Yves David <pierre-yves.david at octobus.net>
> > # Date 1495198021 -7200
> > #      Fri May 19 14:47:01 2017 +0200
> > # Node ID eb7674b12d5a15fc53f10b075dcac7bee91379d2
> > # Parent  3c2a082a590aa8b57693c24b8461c2afdb8d5556
> > # EXP-Topic obscache
> > # Available At https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ 
> > #              hg pull https://www.mercurial-scm.org/repo/users/marmoute/mercurial/  -r eb7674b12d5a
> > obscache: use the obscache to compute the obsolete set
> > 
> > Now that we have a cache and that the cache is kept up to date, we can use it to
> > speeds up the obsolete set computation. This way, we no longer need to load the
> > obsstore for most operation.
> > 
> > On the mercurial-core repository, this provide a significant speed up:
> > 
> > Running "hg  id -r ."
> > - before: 0.630 second (0.56s user 0.06s system 99% cpu 0.630)
> > - after:  0.129 second (0.11s user 0.02s system 98% cpu 0.129)
> > 
> > And the obsstore loading operation disappear from execution profile.
> > 
> > (note: time spent inside the command drop from 0.4 to 0.04s)
> > 
> > To keep the changeset simple it the handling of case were
> > the cache has not been kept up to date is pretty simple. That might introduce a
> > small performance impact during the transition in some case. This will get
> > improved in later changeset.
> > 
> > In addition the cache still needs to parse the full obsstore when updating.
> > There as known way to skip parsing the full obsstore for wrote operation too.
> > This will also get improved later.
> > 
> > diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
> > --- a/mercurial/obsolete.py
> > +++ b/mercurial/obsolete.py
> > @@ -1546,10 +1546,26 @@ def clearobscaches(repo):
> >  def _computeobsoleteset(repo):
> >      """the set of obsolete revisions"""
> >      obs = set()
> > -    getnode = repo.changelog.node
> >      notpublic = repo._phasecache.getrevset(repo, (phases.draft, phases.secret))
> > +    if not notpublic:
> > +        # all changeset are public, none are obsolete
> > +        return obs
> > +
> > +    # XXX There are a couple of case where the cache could not be up to date:
> > +    #
> > +    # 1) no transaction happened in the repository since the upgrade,
> > +    # 2) both old and new client touches that repository
> > +    #
> > +    # recomputing the whole cache in these case is a bit slower that using the
> > +    # good old version (parsing markers and checking them). We could add some
> > +    # logic to fall back to the old way in these cases.
> > +    obscache = repo.obsstore.obscache
> > +    obscache.update(repo) # ensure it is up to date:
> > +    isobs = obscache.get
> > +
> > +    # actually compute the obsolete set
> >      for r in notpublic:
> > -        if getnode(r) in repo.obsstore.successors:
> > +        if isobs(r):
> >              obs.add(r)
> >      return obs
> >  


More information about the Mercurial-devel mailing list