D5296: store: don't read the whole fncache in memory

yuja (Yuya Nishihara) phabricator at mercurial-scm.org
Sat Mar 16 21:51:19 EDT 2019


yuja added a comment.


  > +        for c in iter(functools.partial(fp.read, chunksize), b''):
  >  +            chunk += c
  >  +            try:
  >  +                p = chunk.rindex(b'\n')
  >  +                self.entries |= set(decodedir(chunk[:p + 1]).splitlines())
  
  Nit: you can `entries.update(any_iterable)` which would be slightly faster
  when bucket size changes.
  
  >   Right now this patch does not have any tests. How should I add them?
  >   
  >   1. add a debug config option and pass that to fncachestore and then to fncache
  >   2. have a function which returns the chunk_size, write an extenion in tests which wrap that function and enable that extension in tests
  
  Something like 2. `chunksize` can be a module constant, which can later be
  updated by an extension. That's probably the easiest option.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5296

To: pulkit, #hg-reviewers
Cc: indygreg, yuja, mjpieters, mercurial-devel


More information about the Mercurial-devel mailing list