[PATCH 1 of 1 v3] store: add names of successfully opened revlogs to fncache

[Adrian Buehlmann]

On 2012-04-23 02:57, Mads Kiilerich wrote:
> # HG changeset patch
> # User Mads Kiilerich <mads at kiilerich.com>
> # Date 1335141331 -7200
> # Branch stable
> # Node ID dab6871ccc38b4c1748d67fa4ecf904c3b3018a1
> # Parent  3440757d12850cfafa88a754b34f5d42e62e4c25
> store: add names of successfully opened revlogs to fncache
> 
> fncache permanently drops the names of revlogs that can't be found when the
> datastore iterator stats them, for example when verify walks over all revlogs.
> The only way to re-add a removed revlog name to fncache was to open the revlog
> for write. ...

[..]

> ... Recovery (for example after restoring revlogs from a backup) was
> thus quite hard.

Well. This isn't really true. The fncache file can be reconstructed
today already (i.e. with unpatched mercurial) by doing:

   hg clone --pull oldrepo newrepo

This operation doesn't read the fncache file in oldrepo at all and
creates a new one in newrepo by walking all manifest revisions.

> This change makes fncache more like a cache by adding entries whenever a
> missing revlog is opened.
> 
> Some operations open revlogs for reading without checking if they exist - they
> will now heal fncache. Verify will thus be able to re-create entries that it
> accidentially has removed. That will however only happen after it has
> complained that the revlog is missing. Next time verify is run the problem is
> gone.
> 
> diff --git a/mercurial/store.py b/mercurial/store.py
> --- a/mercurial/store.py
> +++ b/mercurial/store.py
> @@ -372,9 +372,10 @@
>          self.encode = encode
>  
>      def __call__(self, path, mode='r', *args, **kw):
> -        if mode not in ('r', 'rb') and path.startswith('data/'):
> +        opener = self.opener(self.encode(path), mode, *args, **kw)
> +        if path.startswith('data/'):
>              self.fncache.add(path)
> -        return self.opener(self.encode(path), mode, *args, **kw)
> +        return opener

Hmm. What happens if the repo you are reading is on a read-only volume,
like, say a CD-ROM?

What about the write lock on the store?

The fncache file is *not* facultative; operations like 'clone
--uncompressed' are not able to reconstruct a missing fncache file on
the fly like the other caches we have.

So I have some very strong doubts that this is the way to go. Most
certainly not on stable with a last minute discussion during freeze.

>  class fncachestore(basicstore):
>      def __init__(self, path, openertype, encode):
> diff --git a/tests/test-fncache.t b/tests/test-fncache.t
> --- a/tests/test-fncache.t
> +++ b/tests/test-fncache.t
> @@ -55,6 +55,12 @@
>    3 integrity errors encountered!
>    (first damaged changeset appears to be 0)
>    [1]
> +  $ hg verify
> +  checking changesets
> +  checking manifests
> +  crosschecking files in changesets and manifests
> +  checking files
> +  3 files, 3 changesets, 3 total revisions
>    $ cd ..
>  
>  Non store repo:
> diff --git a/tests/test-verify.t b/tests/test-verify.t
> --- a/tests/test-verify.t
> +++ b/tests/test-verify.t
> @@ -90,6 +90,12 @@
>    (first damaged changeset appears to be 0)
>    [1]
>    $ mv _q_u_i_c_k.txt.i .hg/store/data/
> +  $ hg verify
> +  checking changesets
> +  checking manifests
> +  crosschecking files in changesets and manifests
> +  checking files
> +  4 files, 1 changesets, 4 total revisions
>  
>  test revlog corruption
>  

As a general note, I still have problems to see what problem exactly you
are trying to solve with your attempts in this patch thread here.