[PATCH 1 of 2] repair: add functionality to rebuild fncache

[Yuya Nishihara]

On Sat, 20 Jun 2015 20:21:03 -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1434856426 25200
> #      Sat Jun 20 20:13:46 2015 -0700
> # Node ID 421106879340132e8117e0f0b04409204ccc05da
> # Parent  7fdd1782fc4ee9da87d8af13e806dc9055db2c38
> repair: add functionality to rebuild fncache
> 
> Currently, there is no way to recover from a missing or corrupt fncache
> file. This patch changes that.
> 
> The `hg debugrebuildfncache` command is introduced. It ensures the
> fncache is up to date by reconstructing the fncache from all seen files
> encountered during a brute force traversal of the repository's entire
> history.
> 
> The command will add missing entries and will prune excess ones.
> 
> Currently, the command refuses to operate unless the repository has the
> fncache requirement. The command could later grow the ability to
> "upgrade" an existing repository to be fncache enabled.
> 
> When testing this patch on a local clone of the Firefox repository, it
> removed a bunch of entries. Investigation revealed that removed entries
> belonged to empty (0 byte size) .i filelogs. I'm not sure how these
> files came into existence. The stripping code does have logic for
> informing the fncache of removed entries. It's possible the behavior was
> or is buggy or some other process that truncates revlogs isn't updating
> fncache accordingly. Whatever the cause, I thought it worth mentioning,
> because it may indicate the implementation of this command is incorrect.
> 
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -20,9 +20,9 @@ import merge as mergemod
>  import minirst, revset, fileset
>  import dagparser, context, simplemerge, graphmod, copies
>  import random
>  import setdiscovery, treediscovery, dagutil, pvec, localrepo
> -import phases, obsolete, exchange, bundle2
> +import phases, obsolete, exchange, bundle2, repair
>  import ui as uimod
>  
>  table = {}
>  
> @@ -2727,8 +2727,13 @@ def debugrebuilddirstate(ui, repo, rev):
>          repo.dirstate.rebuild(ctx.node(), ctx.manifest())
>      finally:
>          wlock.release()
>  
> + at command('debugrebuildfncache', [], '')
> +def debugrebuildfncache(ui, repo):
> +    """rebuild the fncache file"""
> +    repair.rebuildfncache(ui, repo)
> +
>  @command('debugrename',
>      [('r', 'rev', '', _('revision to debug'), _('REV'))],
>      _('[-r REV] FILE'))
>  def debugrename(ui, repo, file1, *pats, **opts):
> diff --git a/mercurial/repair.py b/mercurial/repair.py
> --- a/mercurial/repair.py
> +++ b/mercurial/repair.py
> @@ -226,4 +226,69 @@ def strip(ui, repo, nodelist, backup=Tru
>              # Remove partial backup only if there were no exceptions
>              vfs.unlink(chgrpfile)
>  
>      repo.destroyed()
> +
> +def rebuildfncache(ui, repo):
> +    """Rebuilds the fncache file from repo history.
> +
> +    Missing entries will be added. Extra entries will be removed.
> +    """
> +    if 'fncache' not in repo.requirements:
> +        raise util.Abort(_('repository does not support fncache'))
> +
> +    lock = repo.lock()
> +    try:
> +        fnc = repo.store.fncache
> +        # Trigger load of fncache.
> +        if 'irrelevant' in fnc:
> +            pass
> +
> +        oldentries = set(fnc.entries)
> +        newentries = set()
> +        seenfiles = set()
> +
> +        repolen = len(repo)
> +        for rev in repo:
> +            ui.progress(_('changeset'), rev, total=repolen)
> +
> +            ctx = repo[rev]
> +            for f in ctx.files():

Perhaps it needs unfiltered repo.