Bug 4110 - malloc crash during basic usage (Mac OS X 10.8, hg 2.8+20131101)
Summary: malloc crash during basic usage (Mac OS X 10.8, hg 2.8+20131101)
Status: RESOLVED INVALID
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: 2.8
Hardware: Macintosh Mac OS
: wish bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-11-21 17:54 UTC by Chris Jerdonek
Modified: 2013-11-29 19:06 UTC (History)
4 users (show)

See Also:
Python Version: ---


Attachments
crash report (29.31 KB, text/plain)
2013-11-21 17:55 UTC, Chris Jerdonek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Jerdonek 2013-11-21 17:54 UTC
On Mac OS X 10.8.5 with hg version 2.8+20131101, running `hg status` (and other basic commands) crashes for me:

    $ hg status
    python(26909) malloc: *** error for object 0x28a660: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Abort trap: 6

Here is the beginning of the Apple Crash Report:

    Process:         Python [22920]
    Path:            /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python
    Identifier:      Python
    Version:         2.5.6 (2.5.6a0)
    Code Type:       X86 (Native)
    Parent Process:  ruby [22254]
    User ID:         502

    PlugIn Path:       /Library/Python/2.7/site-packages/mercurial/parsers.so
    PlugIn Identifier: parsers.so
    PlugIn Version:    ??? (0)

    Date/Time:       2013-11-21 13:31:23.568 -0800
    OS Version:      Mac OS X 10.8.5 (12F45)
    Report Version:  10

I will attach the full crash report separately.
Comment 1 Chris Jerdonek 2013-11-21 17:55 UTC
Created attachment 1749 [details]
crash report
Comment 2 Bryan O'Sullivan 2013-11-21 18:52 UTC
At a guess, this has something to do with the exceedingly ancient version of Python you're using, which is not the system default version. You should be running 2.7.2 or so. Perhaps you're getting a mismatch between Mercurial shared object built against a newer Python and something that's later on frobbing your path to use your ancestral version.
Comment 3 Chris Jerdonek 2013-11-21 18:57 UTC
Thanks for the prompt response, Bryan.  I'm using the latest 2.7 version:

    $ python --version
    Python 2.7.6

But yes, it looks like hg isn't recognizing/finding that though.
Comment 4 Chris Jerdonek 2013-11-21 19:01 UTC
Also, just so you know, this is after installing the latest Mercurial from the downloads page: http://mercurial.selenic.com/downloads/ (Mercurial 2.8 for MacOS X 10.8) though I did have an older version of hg previously.
Comment 5 Bryan O'Sullivan 2013-11-21 19:03 UTC
You're clearly not using Python 2.7 - look at the crash report you pasted into the description of the bug to see that. The Python executable is 2.5, but the shared object is compiled against 2.7.

And Mercurial isn't responsible for finding the version of Python. I see from the crash report that you're invoking Mercurial from inside a ruby script of some sort. Somewhere in the chain of parent processes, the wrong Python is forcibly being run. It's up to you to figure out where in your local environment the problem lies.
Comment 6 Chris Jerdonek 2013-11-21 19:08 UTC
Yes, that crash report had Ruby as the parent process (when I first found the problem), but the problem also occurs when running straight from bash:

    $ python --version
    Python 2.7.6
    $ hg status
    python(28863) malloc: *** error for object 0x2b3660: pointer being freed was not allocated
    *** set a breakpoint in malloc_error_break to debug
    Abort trap: 6

I can send the crash report with bash as the parent process if that would help.  Is there anything that could explain an older version of Python being used with hg when Python 2.7 is the one found on the PATH?
Comment 7 Matt Mackall 2013-11-21 19:25 UTC
Try looking at the contents of "which hg".
Comment 8 Chris Jerdonek 2013-11-21 19:27 UTC
Thanks, Matt.  Yes, I just discovered that, too.  For the record it's because /usr/local/bin/hg begins with:

#!/usr/bin/python

It doesn't use what's on the PATH.
Comment 9 Chris Jerdonek 2013-11-22 17:13 UTC
I do think there's a potential for Mercurial to be doing something better here.  The Mercurial downloads page says it supports Python 2.4 through 2.7, and the 2.5 version in my case was the latest 2.5.

For example, if it's an issue of the shared objects being compiled against 2.7 and not being compatible with 2.5, shouldn't the Mercurial installer check the version of the user's system Python /usr/bin/python before compiling the shared objects?  Or perhaps the hg command and/or installer can warn the user that their system Python might not be compatible.

I've changed my system Python, so it's no longer an issue for me.  Just making a suggestion.
Comment 10 Matt Mackall 2013-11-22 18:24 UTC
The Mercurial build process builds Mercurial against whatever version of Python it is run with. This is done by standard Python distutils and if that's broken on Macs, it's out of scope for us.

The "Mercurial installer", on the other hand, could literally mean half a dozen different things on Mac, some of which install pre-built binaries against the system Python.
Comment 11 Bryan O'Sullivan 2013-11-22 19:32 UTC
I've never come across this problem in spite of years of using Mercurial on Macs. I am sure this is something weird that is unique to Chris's environment.
Comment 12 Chris Jerdonek 2013-11-24 16:35 UTC
> The "Mercurial installer", on the other hand, could literally mean half a dozen different things on Mac,

The installer I used was the Mac OS X 10.8 installer provided on the downloads page: http://mercurial.selenic.com/downloads/

Running the installer again, I see that it says in the dialog box, "This is an OS X 10.8 version of Mercurial that depends on the default Python 2.7 installation."

I'm just brainstorming here, but I wonder if the build process could record for Mercurial what Python versions the build is compatible with (or maybe Mercurial can already read this information from somewhere).  Then if hg is ever run with an incompatible Python (e.g. if the system Python is ever changed), hg could display a message to that effect: "failing fast" and providing a friendlier user experience.
Comment 13 Matt Mackall 2013-11-24 16:43 UTC
We're very unlikely to do anything about this. You're the first person in recorded history to hit this problem and it was self-inflicted, so it comes in well below lots of other issues in priority terms. But: patches accepted.
Comment 14 Chris Jerdonek 2013-11-24 16:48 UTC
That's fine, and I agree re: priority.  I just wanted to record the info for posterity in case a second person ever comes along. :)
Comment 15 Kevin Bullock 2013-11-25 00:06 UTC
For the record, there's a chimp-simple way to reproduce this on any Mac OS that ships with Python 2.7:

 $ VERSIONER_PYTHON_VERSION=2.5 hg status

This makes OS X use its built-in Python 2.5 instead of its default built-in 2.7. Unsurprisingly, trying to load compiled CPython extensions built against 2.7.x into an interpreter in the 2.5.x series results in a bug.

Setting to 'confirmed' but lowering to 'wish', because it might be reasonably easy to add a check in the Mac installer's preflight script to see if the default Python version is the right one and bail otherwise.
Comment 16 Matt Mackall 2013-11-25 12:43 UTC
This is actually supposed to be detected at runtime, and you should get this message:

static char api_version_warning[] =
"Python C API version mismatch for module %.100s:\                              
 This Python has API version %d, module %.100s has version %d.";

..but the Python folks didn't bump the PYTHON_API_VERSION between 2.5 and 2.7 when they changed whatever they changed. So Python is of the opinion that modules built against 2.5 should in fact work with 2.7 and vice versa. Is this awesome? y/n

Still deep in Notourproblemistan.
Comment 17 Chris Jerdonek 2013-11-25 22:44 UTC
Interesting.  So according to Python this was supposed to work with 2.5 and 2.7 after all.

Looking into the reason, I found I can trigger this same crash using the latest Python 2.7.6:

    import mercurial.revlog as revlog
    r = revlog.revlogio()
    r.parseindex(0, 0)

This is because when the string check on the data argument to parsers.parse_index2() fails, parsers.c bails early from index_init():

http://selenic.com/hg/file/af12f58e2aa0/mercurial/parsers.c#l1718

Apparently this leaves the indexObject in a bad, improperly initialized state, because when index_dealloc() is called, it crashes around here when trying to clear self->cache:

http://selenic.com/hg/file/af12f58e2aa0/mercurial/parsers.c#l644

What I'm still curious about is why the original problem happens only with 2.5.  For some reason, PyString_Check(data_obj) in the first source code link above returns false in 2.5 on whatever input is given, but not with 2.6+.  I may investigate that as well.
Comment 18 Chris Jerdonek 2013-11-26 01:17 UTC
Out of curiosity, I figured out why PyString_Check returns false for Python 2.5 when hg is compiled for Python 2.7.  The reason is that Mercurial defines PyString_Check as PyUnicode_Check:

http://selenic.com/hg/file/af12f58e2aa0/mercurial/util.h#l28

And in Python 2.6, PyUnicode_Check was changed to use PyType_FastSubclass:

http://hg.python.org/cpython/rev/d01bd37bf5ce#l9.1

But this new definition doesn't work for the PyUnicode_Type compiled from 2.5 because PyUnicode_Type's tp_flags doesn't have Py_TPFLAGS_UNICODE_SUBCLASS in Python 2.5:

http://hg.python.org/cpython/file/b48e1b48e670/Objects/unicodeobject.c#l8066
Comment 19 Chris Jerdonek 2013-11-26 16:53 UTC
I prepared a patch to fix the crash and will submit after reading the instructions.  I'm also almost done preparing a patch to check the version of the Python interpreter (just need tests).
Comment 20 Chris Jerdonek 2013-11-29 19:06 UTC
Update: the patch to fix the crash was recently applied to stable here:

http://selenic.com/hg/rev/e57c532c3835

And a patch to fail fast if the Python version is wrong was recently submitted here:

http://www.selenic.com/pipermail/mercurial-devel/2013-November/055146.html