Testing Mercurial with ASAN

[Bryan O'Sullivan]

If you haven't come across the Address Sanitizer that's shipped with clang
and gcc the past few years, start here:
https://code.google.com/p/address-sanitizer/wiki/AddressSanitizer

I spent some time building and testing Mercurial against both clang (3.6)
and gcc (4.9) ASAN implementations yesterday. The good, and
unsurprising-to-me news: t's completely clean right now.

Here are some notes, in case anyone wants to reproduce.

Normally, ASAN is built directly into an executable. The standard ASAN
libraries are thus static libraries, not shared objects. This doesn't work
with Mercurial at all, as the C routines in Mercurial are loaded as shared
objects.

Fortunately, gcc supports ASAN-as-DSO out of the box. If you're using
clang, you need to start by building its compiler-rt library with support
for using ASAN as a shared object. (This doesn't seem to be the default
with any distribution of clang.)

cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME/llvm \
  -DCOMPILER_RT_BUILD_SHARED_ASAN=ON ..

Once you've got ASAN DSO support, you need to build a custom Python that
has ASAN support. Here's how I do this with my custom clang build:

export CXX="$HOME/llvm/bin/clang++ -O0 -fsanitize=address"
export CC="$HOME/llvm/bin/clang -O0 -fsanitize=address"
./configure --prefix=$HOME/san

(ASAN was the first of what is now a family of useful sanitizers. For
whatever reason, under both gcc and clang, it's the only one that works as
a DSO, so for the moment you can forget about UBSAN and other toys that
would be nice to play with.)

With the ASAN-enabled Python built, you can build an ASAN-enabled Mercurial
fairly easily:

export LD_PRELOAD=/usr/lib/gcc/i686-redhat-linux/4.9.2/libasan.so
export CC='gcc -g -fsanitize=address'
export PYTHON=$HOME/san/bin/python
make local

Notice the use of LD_PRELOAD (you have to do the same thing with clang) –
without this you'll die due to undefined symbol errors, as the Mercurial
shared objects need the ASAN symbols but are not linked against the ASAN
library.

Finally, go into the tests directory, and use the Python and Mercurial you
just built to run the test suite:

$HOME/san/bin/python run-tests.py --local --jobs $(ncpu)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20150410/de7b90a5/attachment.html>