[PATCH 03 of 10 V2] sparse-revlog: rework the way we enforce chunk size limit

Yuya Nishihara yuya at tcha.org
Thu Nov 15 07:12:44 EST 2018


On Thu, 15 Nov 2018 11:38:41 +0100, Boris Feld wrote:
> # HG changeset patch
> # User Boris Feld <boris.feld at octobus.net>
> # Date 1541782717 -3600
> #      Fri Nov 09 17:58:37 2018 +0100
> # Node ID 20e272f4f88c761b61556b4a4db08890fac21f56
> # Parent  2251b59ed38df5764429e9b924eee57c4a7babae
> # EXP-Topic sparse-perf
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r 20e272f4f88c
> sparse-revlog: rework the way we enforce chunk size limit

Queued up to this, thanks.

I don't wanna read more C today. I'll review the remainder tomorrow or this
weekend.

> > +    == mixed case ==
> +    >>> revlog = _testrevlog(data, snapshot=[0, 1, 2])
> +    >>> list(_slicechunktosize(revlog, range(9), 5))

Fixed range(9) to list(range(9)) for Py3.

>      startrevidx = 0
> -    startdata = revlog.start(revs[0])
>      endrevidx = 0
>      iterrevs = enumerate(revs)
>      next(iterrevs) # skip first rev.
> +    # first step: get snapshots out of the way
>      for idx, r in iterrevs:
>          span = revlog.end(r) - startdata
> -        if span <= targetsize:
> +        snapshot = revlog.issnapshot(r)
> +        if span <= targetsize and snapshot:
>              endrevidx = idx
>          else:
>              chunk = _trimchunk(revlog, revs, startrevidx, endrevidx + 1)

Can you update this "startrevidx..endrevidx" to be a half-open range? It's
confusing that "endrevidx" is inclusive here, but isn't later in this function.

> +    # for the others, we use binary slicing to quickly converge toward valid
> +    # chunks (otherwise, we might end up looking for start/end of many
> +    # revisions). This logic is not looking for the perfect slicing point, it
> +    # focuses on quickly converging toward valid chunks.
> +    nbitem = len(revs)
> +    while (enddata - startdata) > targetsize:
> +        endrevidx = nbitem
> +        if nbitem - startrevidx <= 1:
> +            break # protect against individual chunk larger than limit
> +        localenddata = revlog.end(revs[endrevidx - 1])
> +        span = localenddata - startdata
> +        while (localenddata - startdata) > targetsize:
> +            if endrevidx - startrevidx <= 1:
> +                break # protect against individual chunk larger than limit
> +            endrevidx -= (endrevidx - startrevidx) // 2
> +            localenddata = revlog.end(revs[endrevidx -1])
> +            span = localenddata - startdata

'span' isn't used.


More information about the Mercurial-devel mailing list