D5296: store: don't read the whole fncache in memory

Yuya Nishihara yuya at tcha.org
Mon Mar 18 08:27:20 EDT 2019


Queued, thanks.

> +        self.entries = set()
> +        chunk = b''
> +        for c in iter(functools.partial(fp.read, fncache_chunksize), b''):
> +            chunk += c
> +            try:
> +                p = chunk.rindex(b'\n')
> +                self.entries.update(decodedir(chunk[:p + 1]).splitlines())
> +                chunk = chunk[p + 1:]
> +            except ValueError:
> +                # substring '\n' not found, maybe the entry is bigger than the
> +                # chunksize, so let's keep iterating
> +                pass

We might want to check if the `chunk` is fully consumed. If the file doesn't
end with '\n', which I think is invalid though, the last line would be silently
ignored.


More information about the Mercurial-devel mailing list