D1568: lfs: using workers in lfs prefetch

wlis (Wojciech Lis) phabricator at mercurial-scm.org
Mon Dec 11 19:37:26 EST 2017


wlis added a comment.


  I must have messed up something when running tests previously- probably wrong revision. The tests actually catch the failure above:
  
    [wlis at dev9680.prn1 ~/hg-committed/tests] ./run-tests.py -l test-lfs-test-server.t
    
    --- /home/wlis/hg-committed/tests/test-lfs-test-server.t
    +++ /home/wlis/hg-committed/tests/test-lfs-test-server.t.err
    @@ -43,54 +43,98 @@
       pushing to ../repo2
       searching for changes
       lfs: uploading 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b (12 bytes)
    -  1 changesets found
    -  uncompressed size of bundle content:
    -       * (changelog) (glob)
    -       * (manifests) (glob)
    -       *  a (glob)
    -  adding changesets
    -  adding manifests
    -  adding file changes
    -  added 1 changesets with 1 changes to 1 files
    +  lfs: failed: TypeError("object of type 'file' has no len()",) (remaining retry 5)
    +  lfs: failed: TypeError("object of type 'file' has no len()",) (remaining retry 4)
    +  lfs: failed: TypeError("object of type 'file' has no len()",) (remaining retry 3)
    +  lfs: failed: TypeError("object of type 'file' has no len()",) (remaining retry 2)
    +  lfs: failed: TypeError("object of type 'file' has no len()",) (remaining retry 1)
    +  ** unknown exception encountered, please report by visiting
    +  ** https://mercurial-scm.org/wiki/BugTracker
    +  ** Python 2.7.5 (default, Aug  4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
    +  ** Mercurial Distributed SCM (version 4.4.1+203-4da86512789c)
    +  ** Extensions loaded: lfs
    +  Traceback (most recent call last):
    +    File "/home/wlis/hg-committed/hg", line 41, in <module>
    +      dispatch.run()
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 88, in run
    +      status = (dispatch(req) or 0) & 255
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 177, in dispatch
    +      ret = _runcatch(req)
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 318, in _runcatch
    +      return _callcatch(ui, _runcatchfunc)
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 326, in _callcatch
    +      return scmutil.callcatch(ui, func)
    +    File "/home/wlis/hg-committed/mercurial/scmutil.py", line 154, in callcatch
    +      return func()
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 308, in _runcatchfunc
    +      return _dispatch(req)
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 912, in _dispatch
    +      cmdpats, cmdoptions)
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 667, in runcommand
    +      ret = _runcommand(ui, options, cmd, d)
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 920, in _runcommand
    +      return cmdfunc()
    +    File "/home/wlis/hg-committed/mercurial/dispatch.py", line 909, in <lambda>
    +      d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
    +    File "/home/wlis/hg-committed/mercurial/util.py", line 1188, in check
    +      return func(*args, **kwargs)
    +    File "/home/wlis/hg-committed/mercurial/commands.py", line 4160, in push
    +      opargs=opargs)
    +    File "/home/wlis/hg-committed/mercurial/exchange.py", line 475, in push
    +      _pushbundle2(pushop)
    +    File "/home/wlis/hg-committed/mercurial/exchange.py", line 1023, in _pushbundle2
    +      ret = partgen(pushop, bundler)
    +    File "/home/wlis/hg-committed/mercurial/exchange.py", line 797, in _pushb2ctx
    +      pushop.repo.prepushoutgoinghooks(pushop)
    +    File "/home/wlis/hg-committed/mercurial/util.py", line 3125, in __call__
    +      results.append(hook(*args))
    +    File "/home/wlis/hg-committed/hgext/lfs/wrapper.py", line 263, in prepush
    +      return uploadblobsfromrevs(pushop.repo, pushop.outgoing.missing)
    +    File "/home/wlis/hg-committed/hgext/lfs/wrapper.py", line 254, in uploadblobsfromrevs
    +      uploadblobs(repo, pointers)
    +    File "/home/wlis/hg-committed/hgext/lfs/wrapper.py", line 306, in uploadblobs
    +      remoteblob.writebatch(pointers, repo.svfs.lfslocalblobstore)
    +    File "/home/wlis/hg-committed/hgext/lfs/blobstore.py", line 133, in writebatch
    +      self._batch(pointers, fromstore, 'upload')
    +    File "/home/wlis/hg-committed/hgext/lfs/blobstore.py", line 294, in _batch
    +      for _one, oid in oids:
    +    File "/home/wlis/hg-committed/hgext/lfs/blobstore.py", line 278, in transfer
    +      self._basictransfer(obj, action, localstore)
    +    File "/home/wlis/hg-committed/hgext/lfs/blobstore.py", line 235, in _basictransfer
    +      req = self.urlopener.open(request)
    +    File "/usr/lib64/python2.7/urllib2.py", line 429, in open
    +      req = meth(req)
    +    File "/usr/lib64/python2.7/urllib2.py", line 1152, in do_request_
    +      'Content-length', '%d' % len(data))
    +  TypeError: object of type 'file' has no len()
    +  [1]
    
     Clear the cache to force a download
       $ rm -rf `hg config lfs.usercache`
       $ cd ../repo2
       $ hg update tip -v
    -  resolving manifests
    -  getting a
    -  lfs: downloading 31cf46fbc4ecd458a0943c5b4881f1f5a6dd36c53d6167d5b69ac45149b38e5b (12 bytes)
    -  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
    +  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
    
     When the server has some blobs already
    
       $ hg mv a b
    +  a: $ENOENT$
    +  abort: no files to copy
    +  [255]
       $ echo ANOTHER-LARGE-FILE > c
       $ echo ANOTHER-LARGE-FILE2 > d
       $ hg commit -m b-and-c -A b c d
    +  b: $ENOENT$
    +  abort: failed to mark all new/missing files as added/removed
    +  [255]
       $ hg push ../repo1 -v | grep -v '^  '
       pushing to ../repo1
    -  searching for changes
    -  lfs: need to transfer 2 objects (39 bytes)
    -  lfs: uploading 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 (20 bytes)
    -  lfs: uploading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
    -  1 changesets found
    -  uncompressed size of bundle content:
    -  adding changesets
    -  adding manifests
    -  adding file changes
    -  added 1 changesets with 3 changes to 3 files
    +  no changes found
    
     Clear the cache to force a download
       $ rm -rf `hg config lfs.usercache`
       $ hg --repo ../repo1 update tip -v
    -  resolving manifests
    -  getting b
    -  getting c
    -  lfs: downloading d11e1a642b60813aee592094109b406089b8dff4cb157157f753418ec7857998 (19 bytes)
    -  getting d
    -  lfs: downloading 37a65ab78d5ecda767e8622c248b5dbff1e68b1678ab0e730d5eb8601ec8ad19 (20 bytes)
    -  3 files updated, 0 files merged, 0 files removed, 0 files unresolved
    +  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
    
     Check error message when the remote missed a blob:
    
    
    ERROR: test-lfs-test-server.t output changed
    !
    Failed test-lfs-test-server.t: output changed
    # Ran 1 tests, 0 skipped, 1 failed.
    python hash seed: 1273209229
    [wlis at dev9680.prn1 ~/hg-committed/tests]

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1568

To: wlis, #hg-reviewers, quark, mharbison72
Cc: mharbison72, quark, mercurial-devel


More information about the Mercurial-devel mailing list