[PATCH 1 of 2] repair: add functionality to rebuild fncache

[Adrian Buehlmann]

On 2015-06-21 05:21, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1434856426 25200
> #      Sat Jun 20 20:13:46 2015 -0700
> # Node ID 421106879340132e8117e0f0b04409204ccc05da
> # Parent  7fdd1782fc4ee9da87d8af13e806dc9055db2c38
> repair: add functionality to rebuild fncache
> 
> Currently, there is no way to recover from a missing or corrupt fncache
> file. This patch changes that.

Nit: 'hg clone --pull' can achieve that today. So "no way" isn't exactly
true...

> The `hg debugrebuildfncache` command is introduced. It ensures the
> fncache is up to date by reconstructing the fncache from all seen files
> encountered during a brute force traversal of the repository's entire
> history.
> 
> The command will add missing entries and will prune excess ones.
> 
> Currently, the command refuses to operate unless the repository has the
> fncache requirement.

Perhaps the command could just notify the user and exit (instead of
abort) if the repo does not have the fncache requirement.

Not having fncache could be interpreted as 'nothing needs to be rebuilt
here'.

> The command could later grow the ability to
> "upgrade" an existing repository to be fncache enabled.

Hmm. I'd use 'hg clone --pull' for that. It works in any direction
(upgrade, downgrade). Example:

  hg --config format.usefncache=1 clone --pull A B
  hg --config format.usefncache=0 clone --pull B C

(format.usefncache=1 is default. See fncacheRepoFormat in the wiki or
https://www.selenic.com/mercurial/hgrc.5.html#format)

> When testing this patch on a local clone of the Firefox repository, it
> removed a bunch of entries. Investigation revealed that removed entries
> belonged to empty (0 byte size) .i filelogs. I'm not sure how these
> files came into existence. The stripping code does have logic for
> informing the fncache of removed entries.

That logic was added in f49d60fa40a5 on 2014-03-24 by Durham. Perhaps
those files predate that.

> It's possible the behavior was
> or is buggy or some other process that truncates revlogs isn't updating
> fncache accordingly. Whatever the cause, I thought it worth mentioning,
> because it may indicate the implementation of this command is incorrect.

cd03854a2e06 removed the rewrite logic in store.fncache, which had the
comment:

            # rewrite fncache to remove nonexistent entries
            # (may be caused by rollback / strip)

So, perhaps it was a rollback.

(Perhaps, parts of cd03854a2e06 could be "backed out" and then used for
the implementation if this patch instead...)


> diff --git a/tests/test-debugrebuildfncache.t b/tests/test-debugrebuildfncache.t
> new file mode 100644
> --- /dev/null
> +++ b/tests/test-debugrebuildfncache.t

I think these cases can be added to test-fncache.t instead.