Bug 3092 - largefiles - lfconvert error when converting our repos
Summary: largefiles - lfconvert error when converting our repos
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: unspecified
Hardware: All All
: normal bug
Assignee: Tak
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-08 15:48 UTC by Marc Sanfacon
Modified: 2012-01-11 04:19 UTC (History)
6 users (show)

See Also:
Python Version: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marc Sanfacon 2011-11-08 15:48 UTC
initializing destination lfrepo
** unknown exception encountered, please report by visiting
**  http://mercurial.selenic.com/wiki/BugTracker
** Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit 
(AMD64)]
** Mercurial Distributed SCM (version 2.0)
** Extensions loaded: fetch, color, acl, churn, progress, eol, 
mercurial_keyring
, transplant, relink, mq, extdiff, share, convert, purge, reviewboard, 
CoveoExte
nsions, largefiles
Traceback (most recent call last):
  File "hg", line 42, in <module>
  File "mercurial\dispatch.pyo", line 27, in run
  File "mercurial\dispatch.pyo", line 64, in dispatch
  File "mercurial\dispatch.pyo", line 87, in _runcatch
  File "mercurial\dispatch.pyo", line 684, in _dispatch
  File "mercurial\dispatch.pyo", line 466, in runcommand
  File "mercurial\extensions.pyo", line 184, in wrap
  File "hgext\color.pyo", line 373, in colorcmd
  File "mercurial\dispatch.pyo", line 738, in _runcommand
  File "mercurial\dispatch.pyo", line 692, in checkargs
  File "mercurial\dispatch.pyo", line 681, in <lambda>
  File "mercurial\util.pyo", line 454, in check
  File "hgext\largefiles\lfcommands.pyo", line 87, in lfconvert
  File "hgext\largefiles\lfcommands.pyo", line 308, in 
_lfconvert_addchangeset
  File "hgext\largefiles\reposetup.pyo", line 226, in commitctx
  File "hgext\eol.pyo", line 345, in commitctx
  File "mercurial\localrepo.pyo", line 1174, in commitctx
  File "mercurial\context.pyo", line 1054, in __getitem__
  File "mercurial\context.pyo", line 1096, in filectx
  File "hgext\largefiles\lfcommands.pyo", line 298, in getfilectx
ValueError: need more than 1 value to unpack
Comment 1 Na'Tosha Bard 2011-11-09 07:44 UTC
This is an interesting error.  Is it possible to produce a repository that
can reproduce the problem?
Comment 2 Marc Sanfacon 2011-11-09 08:54 UTC
Not sure how I could do this, Our repo is rather big (~3.5GB).

I can add traces to the calls if you tell me what to help understand the 
problem.
Comment 3 Greg Ward 2011-11-11 20:12 UTC
The first thing to do is disable all your 3rd-party extensions (e.g.
reviewboard) and try again. If that fixes the problem, blame one of them.

The next thing to do is disable all extensions other than largefiles. If
that fixes the problem, it might still be largefiles' fault, but it's still
an interesting data point.

Once that is done, rerun with --debug. If the output is huge, just paste in
the last bit before the crash.
Comment 4 Marc Sanfacon 2011-11-17 06:12 UTC
I disabled all extensions except largefiles and ran in Debug mode. This is the 
last output I got.

converting revisions: 11861/20309 revision (58.40%)
.hgtags
** unknown exception encountered, please report by visiting
**  http://mercurial.selenic.com/wiki/BugTracker
** Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit (AMD64)]
** Mercurial Distributed SCM (version 2.0)
** Extensions loaded: largefiles
Traceback (most recent call last):
  File "hg", line 42, in <module>
  File "mercurial\dispatch.pyo", line 27, in run
  File "mercurial\dispatch.pyo", line 64, in dispatch
  File "mercurial\dispatch.pyo", line 87, in _runcatch
  File "mercurial\dispatch.pyo", line 684, in _dispatch
  File "mercurial\dispatch.pyo", line 466, in runcommand
  File "mercurial\dispatch.pyo", line 738, in _runcommand
  File "mercurial\dispatch.pyo", line 692, in checkargs
  File "mercurial\dispatch.pyo", line 681, in <lambda>
  File "mercurial\util.pyo", line 454, in check
  File "hgext\largefiles\lfcommands.pyo", line 87, in lfconvert
  File "hgext\largefiles\lfcommands.pyo", line 308, in _lfconvert_addchangeset
  File "hgext\largefiles\reposetup.pyo", line 226, in commitctx
  File "mercurial\localrepo.pyo", line 1174, in commitctx
  File "mercurial\context.pyo", line 1054, in __getitem__
  File "mercurial\context.pyo", line 1096, in filectx
  File "hgext\largefiles\lfcommands.pyo", line 298, in getfilectx
ValueError: need more than 1 value to unpack
Comment 5 Marc Sanfacon 2011-11-17 10:30 UTC
On another repo, I got another error:

converting revisions: 503/671 revision (74.96%)
** unknown exception encountered, please report by visiting
**  http://mercurial.selenic.com/wiki/BugTracker
** Python 2.6.6 (r266:84297, Aug 24 2010, 18:13:38) [MSC v.1500 64 bit 
(AMD64)
** Mercurial Distributed SCM (version 2.0)
** Extensions loaded: largefiles
Traceback (most recent call last):
  File "hg", line 42, in <module>
  File "mercurial\dispatch.pyo", line 27, in run
  File "mercurial\dispatch.pyo", line 64, in dispatch
  File "mercurial\dispatch.pyo", line 87, in _runcatch
  File "mercurial\dispatch.pyo", line 684, in _dispatch
  File "mercurial\dispatch.pyo", line 466, in runcommand
  File "mercurial\dispatch.pyo", line 738, in _runcommand
  File "mercurial\dispatch.pyo", line 692, in checkargs
  File "mercurial\dispatch.pyo", line 681, in <lambda>
  File "mercurial\util.pyo", line 454, in check
  File "hgext\largefiles\lfcommands.pyo", line 87, in lfconvert
  File "hgext\largefiles\lfcommands.pyo", line 243, in 
_lfconvert_addchangeset
UnboundLocalError: local variable 'renamed' referenced before assignment
Comment 6 HG Bot 2012-01-09 22:00 UTC
Fixed by http://selenic.com/repo/hg/rev/62098aeb1e15
Levi Bard <levi@unity3d.com>
largefiles: don't reference uninitialized variable (issue3092)

(please test the fix)
Comment 7 HG Bot 2012-01-09 22:00 UTC
Fixed by http://selenic.com/repo/hg/rev/884946c002b8
Levi Bard <levi@unity3d.com>
largefiles: add error checking to tags conversion (issue3092)

(please test the fix)
Comment 8 HG Bot 2012-01-09 22:00 UTC
Fixed by http://selenic.com/repo/hg/rev/3d11da212e30
Levi Bard <levi@unity3d.com>
largefiles: add tests for uncovered codepaths (issue3092)

(please test the fix)
Comment 9 Marc Sanfacon 2012-01-10 14:19 UTC
Just tested on one of the repo and I still have this.

  File "hg", line 42, in <module>
  File "mercurial\dispatch.pyo", line 27, in run
  File "mercurial\dispatch.pyo", line 64, in dispatch
  File "mercurial\dispatch.pyo", line 87, in _runcatch
  File "mercurial\dispatch.pyo", line 684, in _dispatch
  File "mercurial\dispatch.pyo", line 466, in runcommand
  File "mercurial\extensions.pyo", line 184, in wrap
  File "hgext\color.pyo", line 373, in colorcmd
  File "mercurial\dispatch.pyo", line 738, in _runcommand
  File "mercurial\dispatch.pyo", line 692, in checkargs
  File "mercurial\dispatch.pyo", line 681, in <lambda>
  File "mercurial\util.pyo", line 458, in check
  File "hgext\largefiles\lfcommands.pyo", line 85, in lfconvert
  File "hgext\largefiles\lfcommands.pyo", line 241, in 
_lfconvert_addchangeset
UnboundLocalError: local variable 'renamed' referenced before assignment

Testing on the other repo now.
Comment 10 Marc Sanfacon 2012-01-10 16:01 UTC
Forget note #18570, I re-tested it and this time it passed. Looks like I was 
using the wrong hg.

Is it normal that at the end of the process, the repository is a log bigger 
with largefiles than before?

4.4GB without largefiles & 13GB with largefiles.

Thanks
Comment 11 Na'Tosha Bard 2012-01-11 04:19 UTC
What types of binary files do you have as largefiles?  It's possible that
they are compressed in the non-largefile repository, and your converted
repository is storing each file uncompressed -- but a normal clone won't
have the entire history of largefiles.

Try doing a local clone of your largefiles-converted repository and see how
big it is?
Comment 12 Bugzilla 2012-05-12 09:25 UTC

--- Bug imported by bugzilla@serpentine.com 2012-05-12 09:25 EDT  ---

This bug was previously known as _bug_ 3092 at http://mercurial.selenic.com/bts/issue3092