Different (but valid) output in the testsuite

[Javi Merino]

Hi,

The output of some tests in mercurial's testsuite can be different to
what the test expect because of different versions of tools, weird
kernels or slow architectures, but still be valid.  Some examples I've
come across:

1.  With git 1.7.9.1, test-subrepo-git.t fails with:

--- /home/javi/src/mercurial/mercurial/mercurial-2.1.1/tests/test-subrepo-git.t
+++ /home/javi/src/mercurial/mercurial/mercurial-2.1.1/tests/test-subrepo-git.t.err
@@ -482,7 +482,17 @@
 
 Sticky repository, update --clean
   $ hg update --clean tip
-  Previous HEAD position was aa84837... f
+  Warning: you are leaving 2 commits behind, not connected to
+  any of your branches:
+
+    aa84837 f
+    126f2a1 gg
+
+  If you want to keep them by creating a new branch, this may be a good time
+  to do so with:
+
+   git branch new_branch_name aa84837ccfbdfedcdcdeeedc309d73e6eb069edc
+  
   HEAD is now at 32a3438... fff
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ hg id -n

ERROR: /home/javi/src/mercurial/mercurial/mercurial-2.1.1/tests/test-subrepo-git.t output changed
---

2.  In slow machines, the output of test-pull-corruption.t,
test-pull-corruption2.t can change because these tests expect that a
"hg pull" can finish in one second:

--- /build/buildd-mercurial_2.1.1-2-armel-Sg28tg/mercurial-2.1.1/tests/test-pull-pull-corruption.t
+++ /build/buildd-mercurial_2.1.1-2-armel-Sg28tg/mercurial-2.1.1/tests/test-pull-pull-corruption.t.err
@@ -42,9 +42,9 @@
 ... and start another pull before the first one has finished
 
   $ sleep 1
+  $ hg pull ../source2 2>/dev/null
   pulling from ../source1
   requesting all changes
-  $ hg pull ../source2 2>/dev/null
   pulling from ../source2
   adding changesets
   adding manifests

ERROR: /build/buildd-mercurial_2.1.1-2-armel-Sg28tg/mercurial-2.1.1/tests/test-pull-pull-corruption.t output changed
---
https://buildd.debian.org/status/fetch.php?pkg=mercurial&arch=armel&ver=2.1.1-2&stamp=1331263103

A similar thing can happen in test-hup.t:

--- /build/buildd-mercurial_2.1.1-2-mipsel-ZKdaeu/mercurial-2.1.1/tests/test-hup.t
+++ /build/buildd-mercurial_2.1.1-2-mipsel-ZKdaeu/mercurial-2.1.1/tests/test-hup.t.err
@@ -12,9 +12,9 @@
   0
   adding changesets
   $ kill -HUP $P
-  $ wait
   transaction abort!
   rollback completed
   killed!
+  $ wait
   $ echo .hg/* .hg/store/*
   .hg/00changelog.i .hg/journal.bookmarks .hg/journal.branch .hg/journal.desc .hg/journal.dirstate .hg/requires .hg/store .hg/store/00changelog.i .hg/store/00changelog.i.a .hg/store/journal.phaseroots

ERROR: /build/buildd-mercurial_2.1.1-2-mipsel-ZKdaeu/mercurial-2.1.1/tests/test-hup.t output changed
---
https://buildd.debian.org/status/fetch.php?pkg=mercurial&arch=mipsel&ver=2.1.1-2&stamp=1331272650

3. With the Hurd kernel, some error messages are slightly different:

--- /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-acl.t
+++ /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-acl.t.err
@@ -868,7 +868,7 @@
   added 3 changesets with 3 changes to 3 files
   calling hook pretxnchangegroup.acl: hgext.acl.hook
   acl: checking access for user "barney"
-  error: pretxnchangegroup.acl hook raised an exception: [Errno 2] *: '../acl.config' (glob)
+  error: pretxnchangegroup.acl hook raised an exception: [Errno 1073741826] No such file or directory: '../acl.config'
   transaction abort!
   rollback completed
   abort: *: ../acl.config (glob)

ERROR: /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-acl.t output changed

--- /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-bad-pull.t
+++ /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-bad-pull.t.err
@@ -1,7 +1,7 @@
   $ "$TESTDIR/hghave" serve || exit 80
 
   $ hg clone http://localhost:$HGPORT/ copy
-  abort: error: Connection refused
+  abort: error: Address family not supported by protocol
   [255]
 
   $ test -d copy

ERROR: /build/buildd-mercurial_2.1.1-2-hurd-i386-nljf5k/mercurial-2.1.1/tests/test-bad-pull.t output changed
---
https://buildd.debian.org/status/fetch.php?pkg=mercurial&arch=hurd-i386&ver=2.1.1-2&stamp=1331304204

I guess the easy solution for this is to ignore the output of the
commands but, before sending patches I was wondering, is it possible
to add alternative but valid output to the tests?

Cheers,
Javi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20120310/aa0b0e70/attachment.pgp>