D477: revlog: add option to mmap revlog index

[indygreg (Gregory Szorc)]

indygreg requested changes to this revision.
indygreg added a comment.
This revision now requires changes to proceed.


  This seems reasonable as an experimental feature.
  
  I worry about memory mapping all files without regards to size. That seems unnecessary.
  
  Do you know what happens with the underlying file descriptor when it is memory mapped? If an fd stays open, that could lead to fd exhaustion. This can be mitigated by only using mmap for certain files (minimize size, changelog, etc). It's something I'd like an answer to before this is queued.

INLINE COMMENTS

> localrepo.py:602-603
>              self.svfs.options['maxdeltachainspan'] = chainspan
> +        self.svfs.options['mmapindex'] = self.ui.configbool('experimental',
> +                                                            'mmapindex')
>  

As you wrote in the commit message, the benefits of memory mapping likely kick in at some threshold and we'll want to make this behavior conditional on index size. It is tempting to implement that today to facilitate experimentation. If you can do that without introducing extra system calls for the default file open case, do it.

> revlog.py:328
>                  self._maxdeltachainspan = opts['maxdeltachainspan']
> +            if sys.version_info >= (2, 7) and 'mmapindex' in opts:
> +                self._mmapindex = opts['mmapindex']

We only support 2.7 now. So this version sniffing can be removed.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D477

To: mbthomas, #fbhgext, indygreg, #hg-reviewers
Cc: simonfar, mercurial-devel