Partial revlog shrinking

[Greg Ward]

On Sat, Apr 17, 2010 at 4:24 PM, Benoit Boissinot <bboissin at gmail.com> wrote:
> On Sat, Apr 17, 2010 at 9:34 PM, Greg Ward <greg at gerg.ca> wrote:
>>
>> Conclusion #2: my relationship with shrink-revlog is not over.  I
>> *could* just do a full run on the entire manifest now and I'm sure it
>> would shrink down quite a lot.
>
> Did you verify?

Yes: big win.  It shrank 00manifest.d from 170443374 bytes to 27975783
bytes. Curiously, that's *smaller* than it was right after conversion
from CVS.  Did you tweak the default algorithm in shrink-revlog.py
since our last round of changes there?

>>  But that's awfully wasteful: it means
>> reading and writing 105375 manifest logs that will not be reordered in
>> order to reorder the 2569 that really need sorting (shrinking).  And I
>> would like to give my users a command they can run on their
>> now-much-bigger-than-necessary working repos to shrink them; if this
>> command takes an hour when it should take a minute or two, that's
>> rather silly.
>
> It should be fast, since the sort should be stable the already ordered
> entries will be written as-is (like during a pull).

Right you are.  Once again Mercurial surprises me for the better.
Actually, I have shrunk that manifest twice: first run started making
changes around rev 4200 (right after our first branch point), but it
was still tolerably fast to write out the whole manifest -- a couple
of minutes I think.  Then I ran it again this morning to verify that
the sort is stable.  It is, and rewriting the manifest with no changes
is quite fast -- under 30 sec.  That's *with* the overhead of
'progress'.  ;-)

> I think that instead of spending time doing this, you're better off
> investing on parent-delta, which is the long term solution.

Yes.  Unfortunately, that requires serious thinking and effort.  Here
I thought I could spend a rainy Sunday afternoon hacking away on
shrink-revlog.py, but there's nothing left to do there.  It's just
fine.  Sigh.

Greg