Unified tests and graphlog output

Adrian Buehlmann adrian at cadifra.com
Tue Sep 21 09:59:05 CDT 2010


On 21.09.2010 13:35, Adrian Buehlmann wrote:
> On 20.09.2010 21:08, Matt Mackall wrote:
>> On Mon, 2010-09-20 at 18:48 +0200, Adrian Buehlmann wrote:
>>> I found a problem with unified tests containing glog calls (I noticed
>>> this while trying to unify tests-rebase*).
>>>
>>> Assume having the following unified test file (test-test.t):
>>>
>>>
>>>   $ echo "[extensions]" >> $HGRCPATH
>>>   $ echo "graphlog=" >> $HGRCPATH
>>>   $ hg init
>>>   $ echo "foo" > a
>>>   $ hg ci -A -m 1
>>>   adding a
>>>   $ echo "bar" >> a
>>>   $ hg ci -m 2
>>>   $ hg up 0
>>>   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
>>>   $ echo "bla" >> a
>>>   $ hg ci -m 3
>>>   created new head
>>>
>>>   $ hg glog --template '{rev}:{node|short} "{desc}"\n'
>>>   @  2:20c4f79fd7ac "3"
>>>   |
>>>   | o  1:38f24201dcab "2"
>>>   |/
>>>   o  0:2a18120dc1c9 "1"
>>>
>>>
>>> This all looks shiny and nice.
>>>
>>> But...
>>>
>>> Assume you edit the graphlog output part of that file to:
>>>
>>>
>>>   $ hg glog --template '{rev}:{node|short} "{desc}"\n'
>>>   @  2:20c4f79fd7ac "3"
>>>   |
>>>   | o  1:38f24201dcab "2" EVIL HACKERS WERE HERE
>>>   |/
>>>   o  0:2a18120dc1c9 "1"
>>>
>>>
>>> Then running this test in a terminal goes like this:
>>>
>>>
>>>   $ python run-tests.py test-test.t -i
>>>   .
>>>   # Ran 1 tests, 0 skipped, 0 failed.
>>>
>>>
>>> Huh?
>>>
>>> The test doesn't detect that the command output does not match the
>>> expected output?!
>>>
>>> The symptom is that iff there is a '|' character at the beginning of a
>>> line, the rest of the line is irrelevant.
>>>
>>> Yikes.
>>>
>>> I haven't looked at the code of the test framework, but my first guess
>>> is that we are hit here by the regular expression matching feature of
>>> the unified tests.
>>>
>>> Even more evil: if -- alternatively -- you instead change the graphlog
>>> part to:
>>>
>>>
>>>   $ hg glog --template 'HACKED {rev}:{node|short} "{desc}"\n'
>>>   @  2:20c4f79fd7ac "3"
>>>   |
>>>   | o  1:38f24201dcab "2"
>>>   |/
>>>   o  0:2a18120dc1c9 "1"
>>>
>>>
>>> (note the "HACKED" inserted into the template option)
>>>
>>> and then accept the changed test file (test-test.t.err) after running
>>> that test, you will end having:
>>>
>>>
>>>   $ hg glog --template 'HACKED {rev}:{node|short} "{desc}"\n'
>>>   @  HACKED 2:20c4f79fd7ac "3"
>>>   |
>>>   | o  1:38f24201dcab "2"
>>>   |/
>>>   o  HACKED 0:2a18120dc1c9 "1"
>>>
>>>
>>> in your "test-test.t" file (note that "HACKED" has -not- been inserted
>>> for node 1, because it is behind a '|' !).
>>>
>>> Can this be improved, so that unified tests are more graphlog-proof?
>>
>> Something like this seems to work, I'll probably push it out in a bit.
>>
>> diff -r 5f81b1e17787 tests/run-tests.py
>> --- a/tests/run-tests.py	Sun Sep 19 13:12:45 2010 -0500
>> +++ b/tests/run-tests.py	Mon Sep 20 12:16:11 2010 -0500
>> @@ -503,6 +503,9 @@
>>  
>>      def rematch(el, l):
>>          try:
>> +            # hack to deal with graphlog
>> +            if el.startswith('|'):
>> +                el = '\\' + el
>>              return re.match(el, l)
>>          except re.error:
>>              # el is an invalid regex
>>
>>
> 
> I see this has been pushed as b016fc1c0862 ("tests: add hack to avoid
> problem with graphlog in unified tests").
> 
> But b016fc1c0862 only helps for the specific cases reported.
> 
> The following (admittedly) contrived hypothetic error in my
> 'test-test.t' file (from above)
> 
> 
>    $ hg glog --template 'HACKED {rev}:{node|short} "{desc}"\n'
>    @  HACKED 2:20c4f79fd7ac "3"
>    |
>    ||o  1:38f24201dcab "2"
>    |/
>    o  HACKED 0:2a18120dc1c9 "1"
> 
> 
> still falsely passes run-tests.py (note the double || in front of
> revision 1).
> 
> Consider (should fail?)
> 
> 
>   $ echo "the | quick | brown | fox | jumps"
>   the | quick | BLUE | fox | jumps
> 
> 
> which passes run-tests.py (is the fox brown or BLUE?).
> 
> And compare with the real life (from test-rebase-cache):
> 
> 
>   @  8:c11d5b3e9c00
>   |
>   o  7:33c9da881988
>   |
>   | o  6:0e4064ab11a3
>   | |
>   | o  5:5ac035cb5d8f
>   | |
>   | | o  4:8e66061486ee
>   | | |
>   +---o  3:99567862abbe
>   | |
>   | o  2:65a26a4d12f6
>   | |
>   | o  1:0f3f3010ee16
>   |/
>   o  0:1994f17a630e

This seems to work for graphlogs (extending your hack):

diff -r d0a97814b7d7 -r 5e21b5415dff tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -505,7 +505,7 @@ def tsttest(test, options):
         try:
             # hack to deal with graphlog, which looks like bogus regexes
             if el.startswith('|'):
-                el = '\\' + el
+                el = el.replace('|', '\\|')
             return re.match(el, l)
         except re.error:
             # el is an invalid regex


More information about the Mercurial-devel mailing list