[PATCH] wireproto: allow unbundle with hashed heads parameter (issue2126)

[Steven Brown]

On 16 April 2011 23:10, Matt Mackall <mpm at selenic.com> wrote:
> On Sat, 2011-04-16 at 14:58 +0800, Steven Brown wrote:
>> On 16 April 2011 11:26, Shuhei Takahashi <takahashi.shuhei at gmail.com> wrote:
>> > # HG changeset patch
>> > # User Shuhei Takahashi <takahashi.shuhei at gmail.com>
>> > # Date 1302883556 -32400
>> > # Node ID 497aab7b3f8fc038daa6a542f3014477fa8451cf
>> > # Parent  ed22a26fc7c6d7ee436c8b4dc33ff1efc349a522
>> > wireproto: allow unbundle with hashed heads parameter (issue2126)
>> >
>> > Current wire protocol of unbundle sends the list of all heads in the remote
>> > repository to avoid race condition.  This causes "URL too long" error on HTTP
>> > server when the repository has many heads.
>> >
>> > This change allows clients to send SHA1 hash of sorted head hashes instead.
>> > Also, this introduces "unbundlehash" capability to inform them that the server
>> > accepts hashed heads parameter.
>> >
>> > diff -r ed22a26fc7c6 -r 497aab7b3f8f mercurial/wireproto.py
>> > --- a/mercurial/wireproto.py    Fri Apr 15 20:35:59 2011 -0500
>> > +++ b/mercurial/wireproto.py    Sat Apr 16 01:05:56 2011 +0900
>> > @@ -139,7 +139,13 @@
>> >         remote server as a bundle. Return an integer indicating the
>> >         result of the push (see localrepository.addchangegroup()).'''
>> >
>> > -        ret, output = self._callpush("unbundle", cg, heads=encodelist(heads))
>> > +        if self.capable('unbundlehash'):
>> > +            heads = encodelist(['hashed',
>> > +                                util.sha1(''.join(sorted(heads))).digest()])
>> > +        else:
>> > +            heads = encodelist(heads)
>> > +
>> > +        ret, output = self._callpush("unbundle", cg, heads=heads)
>> >         if ret == "":
>> >             raise error.ResponseError(
>> >                 _('push failed:'), output)
>> > @@ -216,7 +222,8 @@
>> >     return "".join(r)
>> >
>> >  def capabilities(repo, proto):
>> > -    caps = 'lookup changegroupsubset branchmap pushkey known getbundle'.split()
>> > +    caps = ('lookup changegroupsubset branchmap pushkey known getbundle ' +
>> > +            'unbundlehash').split()
>> >     if _allowstream(repo.ui):
>> >         requiredformats = repo.requirements & repo.supportedformats
>> >         # if our local revlogs are just revlogv1, add 'stream' cap
>> > @@ -353,7 +360,9 @@
>> >
>> >     def check_heads():
>> >         heads = repo.heads()
>> > -        return their_heads == ['force'] or their_heads == heads
>> > +        heads_hash = util.sha1(''.join(sorted(heads))).digest()
>> > +        return (their_heads == ['force'] or their_heads == heads or
>> > +                their_heads == ['hashed', heads_hash])
>> >
>> >     proto.redirect()
>> >
>> > diff -r ed22a26fc7c6 -r 497aab7b3f8f tests/test-hgweb-commands.t
>> > --- a/tests/test-hgweb-commands.t       Fri Apr 15 20:35:59 2011 -0500
>> > +++ b/tests/test-hgweb-commands.t       Sat Apr 16 01:05:56 2011 +0900
>> > @@ -943,7 +943,7 @@
>> >   $ "$TESTDIR/get-with-headers.py" 127.0.0.1:$HGPORT '?cmd=capabilities'; echo
>> >   200 Script output follows
>> >
>> > -  lookup changegroupsubset branchmap pushkey known getbundle unbundle=HG10GZ,HG10BZ,HG10UN
>> > +  lookup changegroupsubset branchmap pushkey known getbundle unbundlehash unbundle=HG10GZ,HG10BZ,HG10UN
>> >
>> >  heads
>> >
>> > diff -r ed22a26fc7c6 -r 497aab7b3f8f tests/test-unbundlehash.t
>> > --- /dev/null   Thu Jan 01 00:00:00 1970 +0000
>> > +++ b/tests/test-unbundlehash.t Sat Apr 16 01:05:56 2011 +0900
>> > @@ -0,0 +1,31 @@
>> > +
>> > +Test wire protocol unbundle with hashed heads (capability: unbundlehash)
>> > +
>> > +Create a remote repository.
>> > +
>> > +  $ hg init remote
>> > +  $ hg serve -R remote --config web.push_ssl=False --config web.allow_push=* -p $HGPORT -d --pid-file=hg1.pid -E error.log -A access.log
>> > +  $ cat hg1.pid >> $DAEMON_PIDS
>> > +
>> > +Clone the repository and push a change.
>> > +
>> > +  $ hg clone http://localhost:$HGPORT/ local
>> > +  no changes found
>> > +  updating to branch default
>> > +  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
>> > +  $ touch local/README
>> > +  $ hg ci -R local -A -m hoge
>> > +  adding README
>> > +  $ hg push -R local
>> > +  pushing to http://localhost:$HGPORT/
>> > +  searching for changes
>> > +  remote: adding changesets
>> > +  remote: adding manifests
>> > +  remote: adding file changes
>> > +  remote: added 1 changesets with 1 changes to 1 files
>> > +
>> > +Ensure hashed heads format is used.
>> > +The hash here is always the same since the remote repository only has the null head.
>> > +
>> > +  $ cat access.log | grep unbundle
>> > +  * - - [*] "POST /?cmd=unbundle&heads=686173686564+6768033e216468247bd031a0a2d9876d79818f8f HTTP/1.1" 200 - (glob)
>> > _______________________________________________
>> > Mercurial-devel mailing list
>> > Mercurial-devel at selenic.com
>> > http://selenic.com/mailman/listinfo/mercurial-devel
>> >
>>
>> This is similar to a patch that I submitted in January.
>
> I guess that shows how long my memory is. Thanks for looking at this.
>
>> Differences in my patch:
>> a) It generated the unbundle hash on the server, instead of the client.
>>    This makes it trivial to change the value in later releases.
>> b) It added more tests.
>>
>> I now think that a) is unnecessary, but would you please include my tests
>> in your patch?
>>
>> - test-push-cgi.t: successful push, heads in sha1 format (see issue2126)
>> - test-push-http.t: expect success, server lacks the prepush capability
>
> How about I take this patch as it is and you can follow it up with more
> tests?
>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
>

Sure, no problem.