A possible explanation for random "stream ended unexpectedly (got m bytes, expected n)"

[Matt Harbison]

On Tue, 06 Jun 2017 22:30:23 -0400, Matt Harbison <mharbison72 at gmail.com>  
wrote:
>
> Any more info on this?
>
> We are starting to see this for a handful of developers on various  
> repos.  So far, every instance has been cloning on Windows 7, from a  
> CentOS 7 server.  The strange aspect is that it is *very* consistent for  
> the people it strikes.  One developer had it happen every time over a  
> period of days with both http and https.  But then when he tried to get  
> a Wireshark trace, the problem stopped.
>
> Today, a new guy on a fresh install of Windows 7 + updates + Visual  
> Studio got the "0 bytes, 4 expected" message on a very small repo (less  
> than 50 commits I bet).  I upgraded him to 4.2.1 because I remembered  
> there was work in this area to not swallow the actual error, and it  
> coughed up these hairballs:
>
> d:\dev\sw>hg clonevcs sw/toolchain/tpos --traceback
> destination directory: tpos
> requesting all changes
> adding changesets
> adding manifests
> adding file changes
> transaction abort!
> rollback completed
> Traceback (most recent call last):
>    File "mercurial\scmutil.pyo", line 146, in callcatch
>    File "mercurial\dispatch.pyo", line 285, in _runcatchfunc
>    File "mercurial\dispatch.pyo", line 912, in _dispatch
>    File "mercurial\dispatch.pyo", line 648, in runcommand
>    File "mercurial\dispatch.pyo", line 920, in _runcommand
>    File "mercurial\dispatch.pyo", line 909, in <lambda>
>    File "mercurial\util.pyo", line 1080, in check
>    File "mercurial\dispatch.pyo", line 506, in __call__
>    File "mercurial\util.pyo", line 1080, in check
>    File "mercurial\commands.pyo", line 1561, in clone
>    File "mercurial\hg.pyo", line 619, in clone
>    File "mercurial\exchange.pyo", line 1238, in pull
>    File "mercurial\exchange.pyo", line 1378, in _pullbundle2
>    File "mercurial\bundle2.pyo", line 387, in processbundle
> SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1752)
> abort: error: WRONG_VERSION_NUMBER
>
> d:\dev\sw>hg clonevcs sw/toolchain/tpos --traceback --config  
> extensions.mercurial_keyring=!
> http authorization required for
> https://vcs.attotech.com/hg/sw/toolchain/tpos
> realm: SONIA :: SCM Manager
> user: ssiwinski
> password:
> destination directory: tpos
> requesting all changes
> adding changesets
> adding manifests
> adding file changes
> transaction abort!
> rollback completed
> Traceback (most recent call last):
>    File "mercurial\scmutil.pyo", line 146, in callcatch
>    File "mercurial\dispatch.pyo", line 285, in _runcatchfunc
>    File "mercurial\dispatch.pyo", line 912, in _dispatch
>    File "mercurial\dispatch.pyo", line 648, in runcommand
>    File "mercurial\dispatch.pyo", line 920, in _runcommand
>    File "mercurial\dispatch.pyo", line 909, in <lambda>
>    File "mercurial\util.pyo", line 1080, in check
>    File "mercurial\dispatch.pyo", line 506, in __call__
>    File "mercurial\util.pyo", line 1080, in check
>    File "mercurial\commands.pyo", line 1561, in clone
>    File "mercurial\hg.pyo", line 619, in clone
>    File "mercurial\exchange.pyo", line 1238, in pull
>    File "mercurial\exchange.pyo", line 1378, in _pullbundle2
>    File "mercurial\bundle2.pyo", line 387, in processbundle
> SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed  
> or bad record mac (_ssl.c:1752)
> abort: error: DECRYPTION_FAILED_OR_BAD_RECORD_MAC
>
> For him, it worked fine over http.  (The schemes extension is hiding the  
> https:// here.)
>
> The thing I'm going to try tomorrow is to do `hg serve` with the  
> certificate like the test suite, to try to eliminate tomcat and  
> scmmanager as suspects.  I forgot about the generaldelta tidbit, so I'll  
> have to check that tomorrow too.  The repos were created with 3.9 IIRC,  
> and everybody should be on 4.0.2 locally.  (Is there a debug command for  
> that with 3.9?)

Today, we got it to fail over https with a simple `hg serve` from the same  
server (the output there indicated a broken pipe in  
self._sslobj.write(data) ssl.py:689).  The server side has a 3.9 install,  
and the repo is not generaldelta.  The server side output looked like it  
made it further in one attempt than the other.

It also failed over http, served from my Linux development machine and two  
of my Windows 7 machines.  The client side indicated zlib errors.  I'm  
starting to wonder if this is a hardware problem of some sort.  That  
doesn't seem like a satisfying answer though, because apparently they were  
remoted into this same client machine for several hours without an issue,  
and could also clone a much larger repo.  I would think if the problem is  
bad enough to fail every time on this repo, other things would be affected  
too.

> Any other suggestions?  If I can find replacement hardware, I might  
> commandeer the machine.  But since the problem tends to disappear, IDK  
> if it will help.
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel