[Bug 5691] New: Memory use when cloning large repositories

[mercurial-bugs at mercurial-scm.org]

https://bz.mercurial-scm.org/show_bug.cgi?id=5691

            Bug ID: 5691
           Summary: Memory use when cloning large repositories
           Product: Mercurial
           Version: unspecified
          Hardware: PC
                OS: Linux
            Status: UNCONFIRMED
          Severity: feature
          Priority: wish
         Component: Mercurial
          Assignee: bugzilla at mercurial-scm.org
          Reporter: joerg at NetBSD.org
                CC: mercurial-devel at mercurial-scm.org

Test case is the NetBSD src tree, a bundle can be found at
https://cdn.netbsd.org/pub/NetBSD/misc/joerg/src.bundle

Current peak RSS is around 800MB on 64bit platforms and that seems to be a
larger than necessary by a factor of two or three at least. It would be very
nice for older memory-constrained systems to stay within 256MB.

Basic memory profile point to the following places for allocations in
changegroup.py:

330MB line 972: for chunkdata in iter(source.filelogheader, {}):

53MBm, line 378: phases.registernew(repo, tr, targetphase, added)

262MB, line 326: newrevs, newfiles = _addchangegroupfiles(repo, self, revmap,
trp, efiles, needfiles)

160MB, line 310: self._unpackmanifests(repo, revmap, trp, prog, changesets)
263MB, line 297: cgnodes = cl.addgroup(self, csmap, trp,
addrevisioncb=onchangelog)

Line numbers are for hg 4.3.1. The allocations seem to be mostly persistent,
i.e. the memory is only freed after the whole unbundling is done.

-- 
You are receiving this mail because:
You are on the CC list for the bug.