Bug 3743 - abort: unknown compression type '\x82' during conversion
Summary: abort: unknown compression type '\x82' during conversion
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: convert (show other bugs)
Version: earlier
Hardware: PC Linux
: normal bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-12-26 12:09 UTC by John Peacock
Modified: 2017-11-01 18:04 UTC (History)
5 users (show)

See Also:
Python Version: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description John Peacock 2012-12-26 12:09 UTC
I'm hitting the above error while trying to convert a repo (actually extract a subtree into its own repo).  If I run the same command with --debug, I get precious little extra information:

18046 Add deps if database role is chosen.  Refs #8188 (spent 0.25)
source: 3eeddb7b7eeb4370faa45846fde767ae7975d468
converting: 0/18047 revisions (0.00%)
build/packages/shared/installer.pl
getting files: build/packages/shared/installer.pl 1/1 (100.00%)
transaction abort!
rollback completed
run hg sink post-conversion action
run hg source post-conversion action
abort: unknown compression type '\x82'!

I can view the log of that changeset as well as the diff (nothing terribly complicated).  I redid the local clone of the upstream repo to see if that would help and nothing changed.  `hg verify` reports no errors whatsoever.

In this case, I'm willing to entertain a manual work around (hopefully this is the only revision that is a problem).  If there are steps I can take to manually apply this one changeset and then continue with the conversion, please let me know.
Comment 1 John Peacock 2012-12-26 14:09 UTC
(In reply to comment #0)
Additional data point: it is the destination repo that is corrupt, not the source.  I don't understand how this could happen because the destination repo is being created by the convert command itself.
Comment 2 Matt Mackall 2012-12-26 15:13 UTC
What does 'hg verify' say?
Comment 3 John Peacock 2012-12-26 15:23 UTC
Redoing the conversion again ends this way:

...
18045 Add deps if database role is chosen.  Refs #8188 (spent 0.25)             
transaction abort!                                                              
rollback completed                                                              
abort: integrity check failed on 00changelog.i:81!                              

$ hg verify
checking changesets
 changelog@?: rev 79 points to unexpected changeset 80
 (expected 79)
 79: unpacking changeset 1d1662e1947c: unknown compression type '\x82'
 changelog@?: rev 80 points to nonexistent changeset 81
 (expected 80)
 changelog@?: unknown parent 1 6f9fe3288d75 of 6f9fe3288d75
 80: unpacking changeset 6f9fe3288d75: unknown compression type '\x82'
checking manifests
 manifest@?: rev 43 points to unexpected changeset 80
 manifest@?: ae3324ee379d not in changesets
crosschecking files in changesets and manifests
checking files
 build/lib/SpecFile.pm@?: rev 2 points to unexpected changeset 80
 (expected )
 build/lib/SpecFile/linux.pm@?: rev 3 points to unexpected changeset 80
 (expected )
 build/make-pkg@?: rev 8 points to unexpected changeset 80
 (expected )
 build/packages/shared/installer.pl@?: rev 14 points to unexpected changeset 80
 (expected )
 build/packages/shared/msyspg@?: rev 2 points to unexpected changeset 80
 (expected )
 build/refresh_packages@?: rev 2 points to unexpected changeset 80
 (expected )
 build/roll@?: rev 13 points to unexpected changeset 80
 (expected )
 build/support/configure.pl@?: rev 10 points to unexpected changeset 80
 (expected )
107 files, 81 changesets, 161 total revisions
10 warnings encountered!
15 integrity errors encountered!
(first damaged changeset appears to be 79)
Comment 4 John Peacock 2012-12-26 15:37 UTC
(In reply to comment #3)

FYI

$ hg --version
Mercurial Distributed SCM (version 2.4.1)
(see http://mercurial.selenic.com for more information)
Comment 5 Matt Mackall 2012-12-26 15:50 UTC
Is that the source or destination repo? I guess I want to see 'hg verify' for both.
Comment 6 John Peacock 2012-12-26 15:57 UTC
(In reply to comment #5)
Sorry, I should have made that clearer: only the destination repo shows any errors.  The source repo is clean (and local if that has any bearing).

The source repo was a conversion from Subversion originally, so I wouldn't discount that there is some underly structural issue (i.e. syntactically correct but logically insane).
Comment 7 Matt Mackall 2012-12-26 16:29 UTC
Looks to be a duplicate of bug 3335 and/or bug 3693. Are you using a filemap?

Can you try running your convert with (exactly) version 2.1?
Comment 8 Idan Kamara 2012-12-26 16:31 UTC
Can you share the source repo?
Comment 9 John Peacock 2012-12-27 10:28 UTC
(In reply to comment #7)
I didn't have 2.1.0 around (we use internally generated RPM's), but I did have 2.0.2 around and reran the conversion without error.  It even seemed significantly faster than 2.4.1!  I reverted the localrepo.py part of:

  http://selenic.com/repo/hg/rev/ce0ad184f489

but I had the same problem with 2.4.1 (with the same changeset).  No easy fixes for this problem I'm afraid...
Comment 10 John Peacock 2012-12-27 10:30 UTC
(In reply to comment #8)
I'm afraid I cannot share the source repo (proprietary code).  I am willing to rerun the convert with any proposed fixes, however.
Comment 11 Idan Kamara 2012-12-27 10:34 UTC
(In reply to comment #10)

Ok then, try applying this series http://mercurial.markmail.org/thread/lzwtz7zr5x7icyir and rerun.

(if you're having trouble applying, you can wait a few hours and I'll have a repo for you to pull from)
Comment 12 John Peacock 2012-12-27 12:04 UTC
On 12/27/2012 10:34 AM, mercurial-bugs@selenic.com wrote:
> Ok then, try applying this series
> http://mercurial.markmail.org/thread/lzwtz7zr5x7icyir and rerun.

Applied to 2.4.1 and reran without error.

NOTE: 2.0.2 took ~28 minutes and 2.4.1 took ~37 for the same repo.

Thanks!

John
Comment 13 Idan Kamara 2012-12-28 05:50 UTC
(In reply to comment #12)

Great.

Might be interesting to see a --profile of the two to get a sense of where those extra 9 minutes are spent.
Comment 14 HG Bot 2013-01-17 12:30 UTC
Fixed by http://selenic.com/repo/hg/rev/3e4a944c0d04
Idan Kamara <idankk86@gmail.com>
destroyed: keep the filecache in sync with __dict__ (issue3335) (issue3693) (issue3743)

We need to make sure that if X is in the filecache then it's also in the
filecache owner's __dict__, otherwise it will go out of sync:

repo.X                  # first access to X, records stat info in
                        # filecache and updates __dict__
repo._filecache.clear() # removes X from _filecache but it's still in __dict__
repo.invalidate()       # iterates over _filecache and removes entries
                        # from __dict__, but X isn't in _filecache, so
                        # it's kept in __dict__
repo.X                  # X is fetched from __dict__, bypassing the filecache

(please test the fix)