[PATCH 2 of 2] bundle2: add a part to import external bundles

[Pierre-Yves David]

On 10/11/2014 06:39 PM, Mike Hommey wrote:
> On Sat, Oct 11, 2014 at 06:17:57PM -0700, Pierre-Yves David wrote:
>>>   capabilities = {'HG2X': (),
>>>                   'b2x:listkeys': (),
>>>                   'b2x:pushkey': (),
>>>                   'b2x:changegroup': (),
>>> +                'digests': tuple(digester.DIGESTS.keys()),
>>
>> You need to sort this otherwise this the order will be unstable. Also, this
>> may be more appropriate for capabilities in general (and not just bundle2
>> capabilities) but this area is a but fuzzy.
>
> Why would order matter?

because otherwise, the test will be unstable and the testing goat will 
find you.

>>>   def getrepocaps(repo):
>>>       """return the bundle2 capabilities for a given repo
>>>
>>>       Exists to allow extensions (like evolution) to mutate the capabilities.
>>>       """
>>>       caps = capabilities.copy()
>>> @@ -826,16 +889,76 @@ def handlechangegroup(op, inpart):
>>>       if op.reply is not None:
>>>           # This is definitly not the final form of this
>>>           # return. But one need to start somewhere.
>>>           part = op.reply.newpart('b2x:reply:changegroup')
>>>           part.addparam('in-reply-to', str(inpart.id), mandatory=False)
>>>           part.addparam('return', '%i' % ret, mandatory=False)
>>>       assert not inpart.read()
>>>
>>> + at parthandler('b2x:import-bundle')
>>> +def handleimportbundle(op, inpart):
>>> +    """apply a bundle on the repo, given an url and validation information
>>> +
>>> +    The part contains a bundle url, as well as hash digest(s) and size
>>> +    information to validate the downloaded content.
>>> +    The expected format for this part is:
>>> +        <URL>
>>> +        <SIZE in bytes>
>>> +        <DIGEST-TYPE>: <DIGEST-VALUE>
>>
>> Part can have parameters. You should use such parameter instead of encoding
>> the information into the part payload.
>
> I'll have to check how that works.

outgoing_part.addparam(key, value)

value = incoming_part.params[key]

>
>>> +        ...
>>> +    Multiple digest types are supported in the same part, in which case they
>>> +    are all validated.
>>> +    This currently only supports bundle10, but is intended to support bundle20
>>> +    eventually.
>>> +    """
>>
>> Lets play simple and call this b2x:remote-changegroup and support
>> changegroup for now. We will need something wider in the future but this
>> will have to be more complicated.
>>
>>> +    lines = inpart.read().splitlines()
>>> +    try:
>>> +        url = lines.pop(0)
>>> +    except IndexError:
>>> +        raise util.Abort(_('import-bundle format error'))
>>> +    parsed_url = urlparse.urlparse(url)
>>> +    if parsed_url.scheme not in ('http', 'https'):
>>> +        raise util.Abort(_('import-bundle only supports http/https urls'))
>>> +
>>> +    try:
>>> +        size = int(lines.pop(0))
>>> +    except (ValueError, IndexError):
>>> +        raise util.Abort(_('import-bundle format error'))
>>> +    digests = {}
>>> +    for line in lines:
>>> +        if ':' not in line:
>>> +            raise util.Abort(_('import-bundle format error'))
>>> +        k, v = line.split(':', 1)
>>> +        digests[k.strip()] = v.strip()
>>
>> As pointed before, all this code can be replaced by something using
>> parameter.
>>
>>> +    real_part = digestchecker(urllib2.urlopen(url), size, digests)
>>> +
>>> +    # Make sure we trigger a transaction creation
>>> +    #
>>> +    # The addchangegroup function will get a transaction object by itself, but
>>> +    # we need to make sure we trigger the creation of a transaction object used
>>> +    # for the whole processing scope.
>>> +    op.gettransaction()
>>> +    import exchange
>>> +    cg = exchange.readbundle(op.repo.ui, real_part, None)
>>
>> If we focus on changegroup only, we can use the changegroup module directly.
>> Avoiding dancing around circular import issue.
>
> ... except the code to handle the various HG10* headers and setting up
> the corresponding decompression is in exchange.readbundle.

hé, true. The rest (about handling record keep applying).

>
> Mike
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
>

-- 
Pierre-Yves David