[PATCH 01 of 14] tests: roll test-committer.t into test-commit.t

[Adrian Buehlmann]

On 2012-06-08 18:07, Martin Geisler wrote:
> Adrian Buehlmann <adrian at cadifra.com> writes:
> 
>> # HG changeset patch
>> # User Adrian Buehlmann <adrian at cadifra.com>
>> # Date 1339161065 -7200
>> # Node ID d62774e5fdbcf4dd9a170c2ee2efb4dbbc4d7ffc
>> # Parent  35292e67674593ae5a32572eaf7916ab3b09bcda
>> tests: roll test-committer.t into test-commit.t
> 
> I would like to question whether this is a good idea. As I understand
> it, the idea behind reducing the number of tests is that this should
> reduce the *overlap* between tests -- it is wasteful if we have two
> different tests that both start with
> 
>   hg init repo
>   cd repo
>   echo "content" > foo
>   hg add foo
>   hg commit -m "init"
> 
> and then continue with
> 
>   hg log
> 
> in one test and
> 
>   hg cat foo
> 
> in the other.
> 
> Here you move one test into another without reducing the overlap.

That's a correct observation.

> In
> that case I much prefer to have to independent tests: if one fails and
> the other passes, then it's easier to go in and see what is wrong.
> 
> (I even prefer to have independent tests in the general case: I believe
> tests should be orthogonal and while they may have a similar start, they
> should conclude by testing something different. When we cram as much as
> we can into a single test we end up with tests that are convoluted and
> hard to maintain. But that's another story and I'm happy to agree to
> disagree on that.)

Even if I haven't yet reduced the overlap, combining the tests is a
prerequisite and enables reducing the overlap later.

For example, I could prepare a repo at the beginning and then use it for
multiple testblocks later:

  $ cd init refrepo
  < poupulate repo>
  $ cd ..

  $ hg clone -q refrepo a
  $ cd a
  < testblock A >
  $ cd ..

  $ hg clone -q refrepo b
  < testblock B >
  $ cd ..

Also, I think the not yet ideally combined testfile still makes running
the testsuite already faster, as only one big(ger) script has to be
assembled and then executed by the testing process. Altough I haven't
yet made measurements how much faster it actually is.

The argument about making it harder to find failing cases doesn't seem
such a strong one to me. You can easily insert meaningful test comments,
which will help navigate inside the file.

Tests are way more often executed than edited. So I think even a
relatively small speed gain is worth the slightly bigger combined file.

Further on, I don't see much value in having the committer bits really
residing in a separate file. It's again the commit that is tested, so if
someone is looking for tests about commit, it may even be inconvenient
to have to dig into several commit-* files and in the main commit.t file
to see what is where tested.

But of course the whole thing is indeed debatable. Some people will
prefer the faster test, some will prefer the separate files.

In the specific case of test-committer.t I wouldn't expect that anyone
would anytime soon edit this test again. So IMHO, the speed win dominates.