[PATCH 03 of 10 V2] sparse-revlog: rework the way we enforce chunk size limit
Yuya Nishihara
yuya at tcha.org
Thu Nov 15 07:12:44 EST 2018
On Thu, 15 Nov 2018 11:38:41 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.feld at octobus.net>
> # Date 1541782717 -3600
> # Fri Nov 09 17:58:37 2018 +0100
> # Node ID 20e272f4f88c761b61556b4a4db08890fac21f56
> # Parent 2251b59ed38df5764429e9b924eee57c4a7babae
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> # hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 20e272f4f88c
> sparse-revlog: rework the way we enforce chunk size limit
Queued up to this, thanks.
I don't wanna read more C today. I'll review the remainder tomorrow or this
weekend.
> > + == mixed case ==
> + >>> revlog = _testrevlog(data, snapshot=[0, 1, 2])
> + >>> list(_slicechunktosize(revlog, range(9), 5))
Fixed range(9) to list(range(9)) for Py3.
> startrevidx = 0
> - startdata = revlog.start(revs[0])
> endrevidx = 0
> iterrevs = enumerate(revs)
> next(iterrevs) # skip first rev.
> + # first step: get snapshots out of the way
> for idx, r in iterrevs:
> span = revlog.end(r) - startdata
> - if span <= targetsize:
> + snapshot = revlog.issnapshot(r)
> + if span <= targetsize and snapshot:
> endrevidx = idx
> else:
> chunk = _trimchunk(revlog, revs, startrevidx, endrevidx + 1)
Can you update this "startrevidx..endrevidx" to be a half-open range? It's
confusing that "endrevidx" is inclusive here, but isn't later in this function.
> + # for the others, we use binary slicing to quickly converge toward valid
> + # chunks (otherwise, we might end up looking for start/end of many
> + # revisions). This logic is not looking for the perfect slicing point, it
> + # focuses on quickly converging toward valid chunks.
> + nbitem = len(revs)
> + while (enddata - startdata) > targetsize:
> + endrevidx = nbitem
> + if nbitem - startrevidx <= 1:
> + break # protect against individual chunk larger than limit
> + localenddata = revlog.end(revs[endrevidx - 1])
> + span = localenddata - startdata
> + while (localenddata - startdata) > targetsize:
> + if endrevidx - startrevidx <= 1:
> + break # protect against individual chunk larger than limit
> + endrevidx -= (endrevidx - startrevidx) // 2
> + localenddata = revlog.end(revs[endrevidx -1])
> + span = localenddata - startdata
'span' isn't used.
More information about the Mercurial-devel
mailing list