[PATCH 6 of 6] run-tests: use a file for capturing replacements

[Pierre-Yves David]

On 03/28/2015 08:15 PM, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1427575392 25200
> #      Sat Mar 28 13:43:12 2015 -0700
> # Node ID 03d1333662fb83144a7dac9de798c82902ceb30b
> # Parent  f62a75b235644da799d0f43e492e55aca2e66206
> run-tests: use a file for capturing replacements
>
> Currently, the test runner defines a strict set of regular expression
> substitutions that will be performed on the test output. Up to this
> point, if we needed a new substitution in a test, we would modify the
> test harness to add a global substitution. If you did not want to add
> a global substitution, you would need to add a glob or regular
> expression match to each line containing variable output.
>
> As part of developing 3rd party Mercurial extensions, I semi-frequently
> found myself having to add glob and regular expression matching on a
> large number of lines to handle variable output. For example, I have
> a number of tests that run Docker containers and the hostname of the
> started container is variable between test runs. I'd strongly prefer
> to set the hostname to a variable and use that variable as a
> placeholder in the test output. This results in more accurate testing
> than globs and regular expressions, which match potentially more things
> than they should.
>
> This patch changes the replacements mechanism so it is file based.
> Tests can write new replacements to the file defined by the
> $REPLACEMENTS environment variable.

I'm not sure about the variable name. I'm not fan of it, but I've not 
better idea.

> The format of this file is a bit wonky. Entries span 3 lines. The
> reason for this is that other delimiters could appear in regular
> expressions. Newlines are a relatively safe delimiter compared to
> spaces, commas, etc and are easier to write than tabs (escaping
> issues). Still, I'm not convinced this is the best solution. This
> implementation detail should be bikeshedded.

Not super fan either, is there a lightweight standard format we could 
use? (.ini file?)

> I'm sure there is a use case for this feature in an existing test. But
> I was lazy and just implemented a basic test in test-run-tests.t (which
> needs to be done anyway).

I like the overall idea especially the ability to alter it from the 
tests (but I'm not sure how it get reload).

Reading the description it was unclear to me how the existing 
replacement were preserved. But I see that the test runner is 
bootstrapping the file with them.

+1 for going in such direct

-- 
Pierre-Yves David