[PATCH 1 of 8] tests: add a randomized test for pathencode

[Adrian Buehlmann]

On 2012-11-14 23:49, Martin Schröder wrote:
> Am 13.11.2012 23:00, schrieb Bryan O'Sullivan:
>> This is a probabilistic test - it generates different test cases on every
>> run, unless invoked from the command line with a specific seed.
> 
> Is that really a good thing? While covering many test cases in the long
> run, it is not deterministic. A test might fail in one case and succeed
> in another.
> 
> I understand that once a run failed, one could test the failing seed
> explicit in future runs, but I think good code coverage in the first
> place would be better.

The non-determinism is indeed something new for the testsuite. But the
fact that Bryan was able to find two bugs with this test makes it
extremely attractive in my view. [1]

I don't know if this specific test should really be run on every run of
the testsuite or not. If it is run and it fails, perhaps a really
drastic and explicit message should be shown in the log, asking to
carefully note the failing paths and not just rerun the test and ignore
that it failed once.

[1] I previously was just afraid of such bugs, without being able to
find any