How do I diagnose test errors?

[Dirkjan Ochtman]

Paul Moore <p.f.moore <at> gmail.com> writes:
> For example, test-rename failed. I have files test-rename,
> test-rename.err and test-rename.out. Am I right to assume that
> test-rename.out is the expected output, and test-rename.err is what my
> run produced? (And hence that diff between these 2 files shows me
> what's changed as a result of my patch)?

First of all, great work so far on the casefolding!

This is correct, yes.

> Also, as I'm not entirely familiar with what the precise output from
> hg should be, is there an easy way to tell which output in the .out
> file corresponds to which commands in the test (short of following
> through line by line)? For example, one of the failures appears to
> have lines "adding <filename>" which are new. I'm not sure why - I'm
> not clear on the precise rules as to when hg add reports which files
> are added, so I can't tell what this extra output indicates. (But as
> it's "only" user feedback, I'm not sure how important it is - can
> cosmetic issues like this be addressed simply by patching the test
> output to match the new expected output? My instincts say no, as it's
> a change, but frankly, I'd rather concentrate on functionality than on
> what seem to be minor user interface changes).

In general, I think hg reports adds whenever the added file isn't explicitly
named. So bare hg add will report adds, and hg ci -A does too, for example. hg
add file will not report, but hg add dir will. (This is from memory, I think
this is roughly how it works).

As for finding out where exactly the error comes from, you're correct that this
is not always easy to find out. The best you can do is litter the test code with
echo % lines (at least this is how many other tests do it, and how I've done it
in the past) making clear in the output where you are. See my recent
test-mq-strip for an example. Patches adding more such echo statements in tests
that have few of them are very welcome, as far as I'm concerned.

Perhaps splitting up some of the larger tests also wouldn't be such a bad idea,
so if you find any of these, be sure to discuss it.

Cheers,

Dirkjan