D5296: store: don't read the whole fncache in memory

[yuja (Yuya Nishihara)]

yuja added a comment.


  Queued, thanks.
  
  > +        self.entries = set()
  >  +        chunk = b''
  >  +        for c in iter(functools.partial(fp.read, fncache_chunksize), b''):
  >  +            chunk += c
  >  +            try:
  >  +                p = chunk.rindex(b'\n')
  >  +                self.entries.update(decodedir(chunk[:p + 1]).splitlines())
  >  +                chunk = chunk[p + 1:]
  >  +            except ValueError:
  >  +                # substring '\n' not found, maybe the entry is bigger than the
  >  +                # chunksize, so let's keep iterating
  >  +                pass
  
  We might want to check if the `chunk` is fully consumed. If the file doesn't
  end with '\n', which I think is invalid though, the last line would be silently
  ignored.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5296

To: pulkit, #hg-reviewers
Cc: indygreg, yuja, mjpieters, mercurial-devel