D4850: store: pass matcher to store.datafiles() and filter files according to it

[martinvonz (Martin von Zweigbergk)]

martinvonz added inline comments.

INLINE COMMENTS

> store.py:429-437
> +    def datafiles(self, matcher=None):
>          for a, b, size in super(encodedstore, self).datafiles():
>              try:
>                  a = decodefilename(a)
>              except KeyError:
>                  a = None
> +            if matcher and not matcher(_gettrackedpath(a)):

This doesn't seem right to me. Let's say the `matcher` is `rootfilesin:some/dir`, then `matcher('some/dir/foo')` will be True, but `matcher('some')` (the first-level directory) will not be. That seems to mean that the client will not get all the directories it needs.

Maybe this code needs to be made less generic and start walking the directories like other tree-walking algorithms we have do. In repos that use tree manifests for all their commits, we should be able to walk the directories in `.hg/store/meta` and look for files in `.hg/store/data` only for directories found in that walk. However, that only works if all commits use treemanifests. I think it's good enough for now (and maybe forever) to instead pass all the file names into a `util.dirs` object and then walk those directories using `matcher.visitdir()`. For each directory found that way, we would look for the manifest revlog in `.hg/store/meta` and include it if it's found (and ignore it if it's not).

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4850

To: pulkit, #hg-reviewers
Cc: martinvonz, mercurial-devel