Tentative fix for usehttp2=true

Peter Arrenbrecht peter.arrenbrecht at gmail.com
Thu May 12 15:59:05 CDT 2011 

On Thu, May 12, 2011 at 10:16 PM, Augie Fackler <durin42 at gmail.com> wrote:
> On Thu, May 12, 2011 at 2:59 PM, Peter Arrenbrecht
> <peter.arrenbrecht at gmail.com> wrote:
>> Here's another failure with the final pushkey request when cloning thg
>> from bitbucket:
>
> Hm, I've seen something akin to this before, I'll try again tonight. I
> suspect a bug in my chunked transfer encoding code, but I've never had
> any luck finding it.

As I told tonfa on IRC, it happens in the select loop in request(). If
I change the break for "not data" to a continue, it simply continues
looping. So select returns a readable socket (if r) and the recv
proceeds to return nothing (if not data). Weird.
-parren

>
>>
>> $ hg clone https://bitbucket.org/tortoisehg/thg/ /tmp/cloned --config
>> ui.usehttp2=true --config ui.http2debuglevel=debug
>> ...
>> reading remaining 6647 of existing chunk
>> looping with 7 data remaining
>> reading chunk of length 0
>> looping with 2 data remaining
>> added 11737 changesets with 17160 changes to 1363 files (+1 heads)
>> updating to branch default
>> 735 files updated, 0 files merged, 0 files removed, 0 files unresolved
>> sending GET request for
>> /tortoisehg/thg/?cmd=listkeys&namespace=bookmarks to bitbucket.org on
>> port 443
>> socket appears closed in read
>> response read 0 data during _select
>> ** unknown exception encountered, please report by visiting
>> **  http://mercurial.selenic.com/wiki/BugTracker
>> ** Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5]
>> ** Mercurial Distributed SCM (version 1.8.3+149-7ebdbef38406)
>> ** Extensions loaded:
>> Traceback (most recent call last):
>>  File "./hg", line 38, in <module>
>>    mercurial.dispatch.run()
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 16, in run
>>    sys.exit(dispatch(sys.argv[1:]))
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 36, in dispatch
>>    return _runcatch(u, args)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 58, in _runcatch
>>    return _dispatch(ui, args)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 613, in _dispatch
>>    cmdpats, cmdoptions)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 418, in
>> runcommand
>>    ret = _runcommand(ui, options, cmd, d)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 667, in
>> _runcommand
>>    return checkargs()
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 621, in checkargs
>>    return cmdfunc()
>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 610, in <lambda>
>>    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/util.py", line 385, in check
>>    return func(*args, **kwargs)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/commands.py", line 1034, in clone
>>    branch=opts.get('branch'))
>>  File "/home/peo/dev/hg/dev-crew/mercurial/hg.py", line 362, in clone
>>    rb = src_repo.listkeys('bookmarks')
>>  File "/home/peo/dev/hg/dev-crew/mercurial/wireproto.py", line 103, in listkeys
>>    d = self._call("listkeys", namespace=encoding.fromlocal(namespace))
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httprepo.py", line 162, in _call
>>    fp = self._callstream(cmd, **args)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httprepo.py", line 113, in
>> _callstream
>>    resp = self.urlopener.open(req)
>>  File "/usr/lib/python2.6/urllib2.py", line 391, in open
>>    response = self._open(req, data)
>>  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
>>    '_open', req)
>>  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
>>    result = func(*args)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpconnection.py", line
>> 230, in https_open
>>    return self.do_open(self._makesslconnection, req)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpconnection.py", line
>> 201, in do_open
>>    r = h.getresponse()
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpclient/__init__.py",
>> line 644, in getresponse
>>    r._select()
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpclient/__init__.py",
>> line 178, in _select
>>    self._load_response(self._end_headers)
>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpclient/__init__.py",
>> line 261, in _load_response
>>    http_ver, status = hdrs.split(' ', 1)
>> ValueError: need more than 1 value to unpack
>> closed connection to bitbucket.org on 443
>>
>> -parren
>>
>> On Thu, May 12, 2011 at 9:24 PM, Peter Arrenbrecht
>> <peter.arrenbrecht at gmail.com> wrote:
>>> On Thu, May 12, 2011 at 7:12 PM, Augie Fackler <durin42 at gmail.com> wrote:
>>>> For anyone experiencing the ssl defect, I've got a proposed fix out
>>>> for review at http://codereview.appspot.com/4532054/. If that works,
>>>> I'll push it to py-nonblocking-http and mail the corresponding change
>>>> for Mercurial. I'd like someone to test this that was experiencing the
>>>> defect, since I can't reproduce it myself (but I think I got unit
>>>> tests that are about right).
>>>
>>> Works for me. Thanks,
>>> -parren
>>>
>>>> On Wed, May 11, 2011 at 7:34 AM, Peter Arrenbrecht
>>>> <peter.arrenbrecht at gmail.com> wrote:
>>>>> On Wed, May 11, 2011 at 2:20 PM, Peter Arrenbrecht
>>>>> <peter.arrenbrecht at gmail.com> wrote:
>>>>>> On Wed, May 11, 2011 at 2:19 PM, Peter Arrenbrecht
>>>>>> <peter.arrenbrecht at gmail.com> wrote:
>>>>>>> On Wed, May 11, 2011 at 2:17 PM, Augie Fackler <durin42 at gmail.com> wrote:
>>>>>>>>
>>>>>>>> On May 11, 2011, at 7:16 AM, Peter Arrenbrecht wrote:
>>>>>>>>
>>>>>>>>> On Wed, May 11, 2011 at 2:13 PM, Augie Fackler <durin42 at gmail.com> wrote:
>>>>>>>>>> Wow, I'm dumb. I'll get a test and fix (probably what you posted) into the upstream library and then get it imported back into hg.
>>>>>>>>>
>>>>>>>>> Please also look into the read error I mentioned on IRC:
>>>>>>>>> http://paste.pocoo.org/show/386790/
>>>>>>>>> -parren
>>>>>>>>
>>>>>>>> Was this repeatable consistently?
>>>>>>>
>>>>>>> Was and still is. ;)
>>>>>>> -parren
>>>>>>
>>>>>> Saw this with pull/incoming from Bitbucket and on a fresh clone of
>>>>>> oplop from Google Code.
>>>>>
>>>>> Better stack trace here:
>>>>>
>>>>> using https://bitbucket.org/tortoisehg/thg
>>>>> sending capabilities command
>>>>> warning: bitbucket.org certificate with fingerprint
>>>>> 81:2b:08:90:dc:d3:71:ee:e0:7c:b4:75:ce:9b:6c:48:94:56:a1:fe not
>>>>> verified (check hostfingerprints or web.cacerts config setting)
>>>>> comparing with https://bitbucket.org/tortoisehg/thg
>>>>> sending changegroupsubset command
>>>>> Traceback (most recent call last):
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 58, in _runcatch
>>>>>    return _dispatch(ui, args)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 613, in _dispatch
>>>>>    cmdpats, cmdoptions)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 418, in
>>>>> runcommand
>>>>>    ret = _runcommand(ui, options, cmd, d)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 667, in
>>>>> _runcommand
>>>>>    return checkargs()
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 621, in checkargs
>>>>>    return cmdfunc()
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/dispatch.py", line 610, in <lambda>
>>>>>    d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/util.py", line 385, in check
>>>>>    return func(*args, **kwargs)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/commands.py", line 2717, in incoming
>>>>>    ret = hg.incoming(ui, repo, source, opts)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/hg.py", line 472, in incoming
>>>>>    return _incoming(display, subreporecurse, ui, repo, source, opts)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/hg.py", line 431, in _incoming
>>>>>    revs, opts["bundle"], opts["force"])
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/bundlerepo.py", line 333,
>>>>> in getremotechanges
>>>>>    cg = other.changegroupsubset(incoming, onlyheads, 'incoming')
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/wireproto.py", line 123,
>>>>> in changegroupsubset
>>>>>    bases=bases, heads=heads)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/httprepo.py", line 113, in
>>>>> _callstream
>>>>>    resp = self.urlopener.open(req)
>>>>>  File "/usr/lib/python2.6/urllib2.py", line 391, in open
>>>>>    response = self._open(req, data)
>>>>>  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
>>>>>    '_open', req)
>>>>>  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
>>>>>    result = func(*args)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpconnection.py", line
>>>>> 232, in https_open
>>>>>    return self.do_open(self._makesslconnection, req)
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpconnection.py", line
>>>>> 201, in do_open
>>>>>    r = h.getresponse()
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpclient/__init__.py",
>>>>> line 631, in getresponse
>>>>>    r._select()
>>>>>  File "/home/peo/dev/hg/dev-crew/mercurial/httpclient/__init__.py",
>>>>> line 168, in _select
>>>>>    data = self.sock.recv(INCOMING_BUFFER_SIZE)
>>>>>  File "/usr/lib/python2.6/ssl.py", line 215, in recv
>>>>>    return self.read(buflen)
>>>>>  File "/usr/lib/python2.6/ssl.py", line 136, in read
>>>>>    return self._sslobj.read(len)
>>>>> SSLError: [Errno 2] _ssl.c:1331: The operation did not complete (read)
>>>>> abort: _ssl.c:1331: The operation did not complete (read)
>>>>>
>>>>
>>>
>>
>

More information about the Mercurial-devel mailing list