Bug 3858 - Exception on rebase
Summary: Exception on rebase
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: rebase (show other bugs)
Version: earlier
Hardware: All All
: normal bug
Assignee: Bugzilla
URL:
Keywords:
: 3864 3944 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-03-16 13:28 UTC by Jan Rysavy
Modified: 2017-11-01 18:05 UTC (History)
11 users (show)

See Also:
Python Version: ---


Attachments
State before rebase (26.02 KB, image/png)
2013-03-16 13:28 UTC, Jan Rysavy
Details
Graph with Phase column visilbe (25.95 KB, image/png)
2013-04-08 06:17 UTC, Jan Rysavy
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Rysavy 2013-03-16 13:28 UTC
Created attachment 1717 [details]
State before rebase

I can reproduce following exception on our repository. It is simple repository - no subrepos, no largefiles extension.  Trying to rebase rev 2490 to current rev 2504 (see attached graph.png).

TortoiseHg version 2.7.1 with Mercurial-2.5.2, Python-2.7.3, PyQt-4.9.6, Qt-4.8.4.

** unknown exception encountered, please report by visiting
** http://mercurial.selenic.com/wiki/BugTracker
** Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)]
** Mercurial Distributed SCM (version 2.5.2)
** Extensions loaded: rebase
Traceback (most recent call last):
  File "hg", line 42, in <module>
  File "mercurial\dispatch.pyo", line 28, in run
  File "mercurial\dispatch.pyo", line 65, in dispatch
  File "mercurial\dispatch.pyo", line 88, in _runcatch
  File "mercurial\dispatch.pyo", line 743, in _dispatch
  File "mercurial\dispatch.pyo", line 514, in runcommand
  File "mercurial\dispatch.pyo", line 833, in _runcommand
  File "mercurial\dispatch.pyo", line 804, in checkargs
  File "mercurial\dispatch.pyo", line 740, in <lambda>
  File "mercurial\util.pyo", line 475, in check
  File "hgext\rebase.pyo", line 327, in rebase
  File "hgext\rebase.pyo", line 732, in clearrebased
  File "mercurial\repair.pyo", line 183, in strip
  File "mercurial\localrepo.pyo", line 58, in wrapper
  File "mercurial\localrepo.pyo", line 1426, in destroyed
  File "mercurial\branchmap.pyo", line 75, in updatecache
  File "mercurial\localrepo.pyo", line 634, in branchmap
  File "mercurial\branchmap.pyo", line 80, in updatecache
  File "mercurial\branchmap.pyo", line 192, in update
  File "mercurial\ancestor.pyo", line 230, in __iter__
  File "mercurial\changelog.pyo", line 204, in parentrevs
IndexError: 2488
Comment 1 Siddharth Agarwal 2013-03-16 20:25 UTC
http://hg.intevation.org/mercurial/file/2.5.2/mercurial/changelog.py#l204

Do you have the Changeset Evolution extension enabled? That code shouldn't kick in unless you do.

Something else to check: inside the .hg/store directory, do you have a file called obsstore?
Comment 2 Jan Rysavy 2013-03-17 01:58 UTC
The only enabled extension I have in TortoiseHG Settings / mercurial.ini is "rebase".

There is no "obsstore" file inside .hg\store directory.
Comment 3 Matt Mackall 2013-03-22 13:41 UTC
*** Bug 3864 has been marked as a duplicate of this bug. ***
Comment 4 Matt Mackall 2013-03-22 14:03 UTC
My guess:

changeset:   18395:904b7109938e
user:        Pierre-Yves David <pierre-yves.david@logilab.fr>
date:        Wed Jan 16 00:09:26 2013 +0100
summary:     destroyed: drop complex branchcache rebuilt logic
Comment 5 Bryan O'Sullivan 2013-04-01 17:23 UTC
I've seen this happen at Facebook, too.
Comment 6 Pierre-Yves David 2013-04-02 04:29 UTC
Probably related to http://bz.selenic.com/show_bug.cgi?id=3827 (fixed recently)
Comment 7 Bryan O'Sullivan 2013-04-05 13:41 UTC
(In reply to comment #6)

Pierre-Yves, does that mean that you think this bug is also likely to have been fixed by that commit?
Comment 8 Pierre-Yves David 2013-04-08 05:09 UTC
It might be fixed, but I'm not sure. Can Jan Rysavy details the command used for rebasing

Did you use --source or --base ?
Comment 9 Jan Rysavy 2013-04-08 06:03 UTC
U used following command: hg rebase --source 2490

I found this problem with TortoiseHG and originally reported it there:
https://bitbucket.org/tortoisehg/thg/issue/2483/freezed-in-the-rebase-dialog-box
Comment 10 Pierre-Yves David 2013-04-08 06:11 UTC
thanks for the quick answer. Do you still have the repo handy. What is the phase of this 2488 changesets ?
Comment 11 Jan Rysavy 2013-04-08 06:17 UTC
Created attachment 1722 [details]
Graph with Phase column visilbe
Comment 12 Jan Rysavy 2013-04-08 06:18 UTC
See attached screenshot, does it help?
Yes, I have 7-Zip archive of this repo state.
Comment 13 Pierre-Yves David 2013-04-08 06:23 UTC
What state is recorded in your archive? Does it allows to reproduce the issue ?
Comment 14 Jan Rysavy 2013-04-08 06:28 UTC
Yes, I can reproduce this problem.
Comment 15 Pierre-Yves David 2013-04-08 06:40 UTC
Marvelous. Can you upload the archive somewhere and provide a minimal script to reproduce the issue ?
Comment 16 Jan Rysavy 2013-04-08 06:43 UTC
Unfortunately it is not possible. It is closed source commercial software, I'm sorry. Please let me know if I can do any tests, it is not a problem.
Comment 17 Pierre-Yves David 2013-04-08 06:49 UTC
1) Can you check the rebase with the latest mercurila release (2.5.4)

2) Can you run the rebase with --debugger goes up withing the branchmap.update method (type "up" twice) then show up the content of `repo.filtername` and `repo.changelog.filteredrevs`
Comment 18 Jan Rysavy 2013-04-08 07:18 UTC
rebase --source 2490
saved backup bundle to X:\Test\ProtechCD\.hg\strip-backup\aa75bfb36958-backup.hg

** unknown exception encountered, please report by visiting
** http://mercurial.selenic.com/wiki/BugTracker
** Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)]
** Mercurial Distributed SCM (version 2.5.4)
** Extensions loaded: rebase
Traceback (most recent call last):
  File "hg", line 38, in <module>
  File "mercurial\dispatch.pyc", line 28, in run
  File "mercurial\dispatch.pyc", line 65, in dispatch
  File "mercurial\dispatch.pyc", line 88, in _runcatch
  File "mercurial\dispatch.pyc", line 743, in _dispatch
  File "mercurial\dispatch.pyc", line 514, in runcommand
  File "mercurial\dispatch.pyc", line 833, in _runcommand
  File "mercurial\dispatch.pyc", line 804, in checkargs
  File "mercurial\dispatch.pyc", line 740, in <lambda>
  File "mercurial\util.pyc", line 475, in check
  File "hgext\rebase.pyc", line 328, in rebase
  File "hgext\rebase.pyc", line 739, in clearrebased
  File "mercurial\repair.pyc", line 183, in strip
  File "mercurial\localrepo.pyc", line 58, in wrapper
  File "mercurial\localrepo.pyc", line 1425, in destroyed
  File "mercurial\branchmap.pyc", line 75, in updatecache
  File "mercurial\localrepo.pyc", line 634, in branchmap
  File "mercurial\branchmap.pyc", line 80, in updatecache
  File "mercurial\branchmap.pyc", line 192, in update
  File "mercurial\ancestor.pyc", line 230, in __iter__
  File "mercurial\changelog.pyc", line 204, in parentrevs
IndexError: 2488

Regarding your second question: I don't know Python nor Python debugger, I will need more detailed guidance here. I did:

X:\Test\ProtechCD>"C:\Program Files\Mercurial\hg.exe" rebase --source 2490 --deb
ugger
entering debugger - type c to continue starting hg or h for help
> x:\test\protechcd\mercurial\dispatch.pyc(87)_runcatch()
(Pdb) up
> x:\test\protechcd\mercurial\dispatch.pyc(65)dispatch()
(Pdb) up
> x:\test\protechcd\mercurial\dispatch.pyc(28)run()

What should I do next exactly?
Comment 19 Matt Mackall 2013-04-08 11:56 UTC
The debugger wants the 'c' command to start running. It will drop you back to a prompt when something breaks. At that point, you can use 'u' to go up to the desired stack frame and 'p somevar' to print a variable.



It also might be worth trying to bisect the problem (see hg help bisect). Grab a copy of Hackable Mercurial for Windows here:

http://mercurial.selenic.com/wiki/HackableMercurial

..that will make it easy to test a sequence of changes.
Comment 20 Jan Rysavy 2013-04-08 13:18 UTC
Thank you! Requested values:

X:\Test\ProtechCD>"C:\Program Files\Mercurial\hg.exe" rebase --source 2490 --deb
ugger
entering debugger - type c to continue starting hg or h for help
> x:\test\protechcd\mercurial\dispatch.pyc(87)_runcatch()
(Pdb) c
saved backup bundle to X:\Test\ProtechCD\.hg\strip-backup\aa75bfb36958-backup.hg

Traceback (most recent call last):
  File "mercurial\dispatch.pyc", line 87, in _runcatch
  File "mercurial\dispatch.pyc", line 743, in _dispatch
  File "mercurial\dispatch.pyc", line 514, in runcommand
  File "mercurial\dispatch.pyc", line 833, in _runcommand
  File "mercurial\dispatch.pyc", line 804, in checkargs
  File "mercurial\dispatch.pyc", line 740, in <lambda>
  File "mercurial\util.pyc", line 475, in check
  File "hgext\rebase.pyc", line 328, in rebase
  File "hgext\rebase.pyc", line 739, in clearrebased
  File "mercurial\repair.pyc", line 183, in strip
  File "mercurial\localrepo.pyc", line 58, in wrapper
  File "mercurial\localrepo.pyc", line 1425, in destroyed
  File "mercurial\branchmap.pyc", line 75, in updatecache
  File "mercurial\localrepo.pyc", line 634, in branchmap
  File "mercurial\branchmap.pyc", line 80, in updatecache
  File "mercurial\branchmap.pyc", line 192, in update
  File "mercurial\ancestor.pyc", line 230, in __iter__
  File "mercurial\changelog.pyc", line 204, in parentrevs
IndexError: 2488
> x:\test\protechcd\mercurial\changelog.pyc(204)parentrevs()
(Pdb) up
> x:\test\protechcd\mercurial\ancestor.pyc(230)__iter__()
(Pdb) up
> x:\test\protechcd\mercurial\branchmap.pyc(192)update()
(Pdb) p repo.filtername
'immutable'
(Pdb) p repo.changelog.filteredrevs
frozenset([2488, 2490, 2491, 2492])
(Pdb)
Comment 21 Siddharth Agarwal 2013-04-12 18:29 UTC
What do the following commands say?

p bheadrevs
p newheadrevs
p latest

This smells like a cache invalidation issue. It's still not fixed, as we've been hitting it occasionally at FB where we run tip of crew. Unfortunately I haven't been able to get a reproducible test case for this.
Comment 22 Jan Rysavy 2013-04-13 03:37 UTC
(Pdb) p bheadrevs
[392, 2487, 2489, 2495, 2500, 2501, 2502, 2503, 2504]
(Pdb) p newheadrevs
[2489, 2495, 2500, 2501, 2502, 2503, 2504]
(Pdb) p latest
2504
Comment 23 Jan Rysavy 2013-04-13 04:02 UTC
Maybe we can arrange a teamviewer session so you can debug this problem on my machine?
Comment 24 Nikolaj Sjujskij 2013-04-15 04:20 UTC
I've just hit the issue on mercurial repo. Let me describe what has happened, while I have terminal session open (sorry for i18ned messages):

I had two draft changeset bookmarked "zsh":

 % hg out

набор изм-й:   18903:35111d556de8
автор:         Nikolaj Sjujskij <sterkrig@myopera.com>
дата:          Mon Apr 08 16:51:38 2013 +0400
сводка:        zsh_completion: make use of `debuglabelcomplete` command

набор изм-й:   18904:bd30c1ae9357
закладка:      zsh
метка:         tip
автор:         Nikolaj Sjujskij <sterkrig@myopera.com>
дата:          Tue Apr 09 19:58:04 2013 +0400
сводка:        zsh_completion: make use of `debugpathcomplete` command

They seem to have been branched off 8c0a7eeda06d (on Fri, Apr 12 in case that matters for cache invalidation).

Today (on Apr 15) I pulled in changes from main with 35111d556de8 being accepted as 5df602551eea. Naturally, I decided to rebase not yet finished 'bd30c1ae9357':

 % hg rebase -s 18904
bundle saved to /home/sterkrig/dev/mercurial/.hg/strip-backup/bd30c1ae9357-backup.hg
Traceback similar to already posted: http://bpaste.net/show/91435/

 WAT?

 % hg out            

набор изм-й:   18903:35111d556de8
закладка:      zsh
автор:         Nikolaj Sjujskij <sterkrig@myopera.com>
дата:          Mon Apr 08 16:51:38 2013 +0400
сводка:        zsh_completion: make use of `debuglabelcomplete` command

набор изм-й:   18944:4d9daa13d870
метка:         tip
автор:         Nikolaj Sjujskij <sterkrig@myopera.com>
дата:          Tue Apr 09 19:58:04 2013 +0400
сводка:        zsh_completion: make use of `debugpathcomplete` command

Revision numbers have changed.

`hg rebase -s 18944 -d @` went all right.

I have this bundle, but:
 % hg in ../bd30c1ae9357-backup.hg
comparing with ../bd30c1ae9357-backup.hg
abort: 00changelog.i@35111d556de8: unknown parent!

Even within *another* repo where I pulled all the other strip-backups created today, which were not much help:

 % hg out                         
changeset:   18943:97b2b6d02687
parent:      18912:4e1ae55e63ef
user:        Nikolaj Sjujskij <sterkrig@myopera.com>
date:        Tue Apr 09 19:58:04 2013 +0400
summary:     zsh_completion: make use of `debugpathcomplete` command

changeset:   18944:4d9daa13d870
tag:         tip
parent:      18942:6891e361bec6
user:        Nikolaj Sjujskij <sterkrig@myopera.com>
date:        Tue Apr 09 19:58:04 2013 +0400
summary:     zsh_completion: make use of `debugpathcomplete` command
Comment 25 Nikolaj Sjujskij 2013-04-15 04:48 UTC
I'm running current default tip (updated regularly), and don't use evolve.
But those two changesets were `histedit`ed several times.
Comment 26 Pierre-Yves David 2013-04-15 08:58 UTC
I've succesfully built a stable repo. Expect detail and probable fix soon.
Comment 27 Pierre-Yves David 2013-04-15 11:16 UTC
fix send to the list
Comment 28 HG Bot 2013-04-16 13:15 UTC
Fixed by http://selenic.com/repo/hg/rev/31bcc5112191
Pierre-Yves David <pierre-yves.david@logilab.fr>
destroyed: invalidate phraserevs cache in all case (issue3858)

When revisions are destroyed, the `phaserevs` cache becomes invalid in most case.
This cache hold a `{rev => phase}` mapping and revision number most likely
changed.

Since 1c8e0d6ac3b0, we filter unknown phases' roots after changesets
destruction.  When some roots are filtered the `phaserevs` cache is invalidated.
But not if none root where destroyed.

We now invalidate the cache in all case filtered root or not.

This bug was a bit tricky to reproduce as in most case we either:
* rebase a set a draft changeset including root (phaserev invalidated)
* strip tip-most changesets (no re-numbering of revision)

Note that the invalidation of `phaserevs` are not strictly needed when only
tip-most part of the history have been destroyed. But I do not expect the
overhead to be significant.

(please test the fix)
Comment 29 Jan Rysavy 2013-04-17 12:59 UTC
Could you please point me to Mercurial nightly build (already compiled version) for Windows?

http://code.google.com/p/mercurial-nightly-builds/downloads/list seems outdated.
Comment 30 Matt Mackall 2013-04-17 13:15 UTC
Try this:

http://mercurial.selenic.com/wiki/HackableMercurial
Comment 31 Jan Rysavy 2013-04-17 13:31 UTC
Doesn't look like nightly build. I will wait for next release of Mercurial and let you know.
Comment 32 Jan Rysavy 2013-04-23 13:54 UTC
The bug is fixed in Mercurial 2.6-RC. Thank you!
Comment 33 Rémi Cardona 2013-04-24 03:01 UTC
Many thanks to all involved!
Comment 34 Matt Mackall 2013-05-24 10:28 UTC
*** Bug 3944 has been marked as a duplicate of this bug. ***
Comment 35 Pierre-Yves David 2014-11-04 13:46 UTC
reopened with no explanation, reclosing.