Bug 6122 - OpenBSD and libressl: some strings in the tests need an update
Summary: OpenBSD and libressl: some strings in the tests need an update
Status: RESOLVED ARCHIVED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: unspecified
Hardware: PC Other
: wish bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-04-22 11:58 UTC by Juan Francisco Cantero Hurtado
Modified: 2019-10-26 00:03 UTC (History)
2 users (show)

See Also:
Python Version: ---


Attachments
mercurial-tests.txt (12.49 KB, text/plain)
2019-04-22 11:58 UTC, Juan Francisco Cantero Hurtado
Details
Fix compatibility problems with OpenBSD grep (1.60 KB, patch)
2019-05-18 15:08 UTC, Juan Francisco Cantero Hurtado
Details | Diff
Fix compatibility problems with OpenBSD grep, take 2 (1.57 KB, patch)
2019-05-20 18:21 UTC, Juan Francisco Cantero Hurtado
Details | Diff
Fix compatibility problems with OpenBSD grep, take 3 (2.50 KB, patch)
2019-05-21 13:38 UTC, Juan Francisco Cantero Hurtado
Details | Diff
output of test-https (4.76 KB, text/plain)
2019-05-24 19:24 UTC, Juan Francisco Cantero Hurtado
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Juan Francisco Cantero Hurtado 2019-04-22 11:58 UTC
Created attachment 2041 [details]
mercurial-tests.txt

I've attached the output of the tests running with Mercurial 5.0rc0 on OpenBSD -current amd64. Some tests need an update to match the output of libressl.

Also, there are a couple of tests unrelated to libressl which need update the matches. "test-lfs-serve.t" checks the output of grep but not of ggrep (GNU grep binary on OpenBSD) and "test-clonebundles.t" needs add the output of the command on OpenBSD.

I'm reporting this bug quite late in the cycle, so feel free to postpone it for 5.1. There is no hurry.
Comment 1 Augie Fackler 2019-05-17 10:55 UTC
I'm disinclined to fix the "ggrep" thing - can't you just use regular grep on your system?

I'm not sure what the deal is with the "wrong ssl version" instead of "unuspported protocol" - is libressl doing something funny with supported TLS versions?
Comment 2 Augie Fackler 2019-05-17 11:14 UTC
Got confirmation from a friend on IRC that libressl just has different error string ideas, patch incoming for that. Still ignoring the ggrep thing for now.
Comment 3 Juan Francisco Cantero Hurtado 2019-05-18 15:06 UTC
OpenBSD grep is not compatible with the | in the grep argument.

This is the output when I use the system grep for test-remotefilelog-gcrepack.t:

   $ find $CACHEDIR | sort | grep ".datapack\|.histpack"                            
-  $TESTTMP/hgcache/master/packs/7bcd2d90b99395ca43172a0dd24e18860b2902f9.histpack
-  $TESTTMP/hgcache/master/packs/dc8f8fdc76690ce27791ce9f53a18da379e50d37.datapack  
+  [1]

I've attached a patch which changes the grep command to something compatible using two -e patterns. Here is the output of both cases:

$ cat test.txt | grep ".datapack\|.histpack"                          
$ cat test.txt | ggrep ".datapack\|.histpack"
$TESTTMP/hgcache/master/packs/7bcd2d90b99395ca43172a0dd24e18860b2902f9.histpack
$TESTTMP/hgcache/master/packs/dc8f8fdc76690ce27791ce9f53a18da379e50d37.datapack
$ cat test.txt | grep -e ".datapack" -e ".histpack"
$TESTTMP/hgcache/master/packs/7bcd2d90b99395ca43172a0dd24e18860b2902f9.histpack
$TESTTMP/hgcache/master/packs/dc8f8fdc76690ce27791ce9f53a18da379e50d37.datapack
$ cat test.txt | ggrep -e ".datapack" -e ".histpack"
$TESTTMP/hgcache/master/packs/7bcd2d90b99395ca43172a0dd24e18860b2902f9.histpack
$TESTTMP/hgcache/master/packs/dc8f8fdc76690ce27791ce9f53a18da379e50d37.datapack
Comment 4 Juan Francisco Cantero Hurtado 2019-05-18 15:08 UTC
Created attachment 2045 [details]
Fix compatibility problems with OpenBSD grep
Comment 5 Augie Fackler 2019-05-18 21:26 UTC
Can we just replace `grep` with `egrep` in this particular case? Does that work?

(Curiously, bsdgrep from FreeBSD seems okay with the provided pattern.)
Comment 6 Juan Francisco Cantero Hurtado 2019-05-20 18:21 UTC
Created attachment 2048 [details]
Fix compatibility problems with OpenBSD grep, take 2
Comment 7 Juan Francisco Cantero Hurtado 2019-05-20 18:31 UTC
I've tested "cat test.txt | grep ".datapack\|.histpack"" on a few unixes. Linux, FreeBSD, NetBSD and BusyBox work with the original grep. OpenBSD and Solaris 11 fail.

The patch attached fixes the problem changing to egrep and removing "\" from the string (which is still problematic for OpenBSD egrep). The new egrep pattern worked everywhere.

Credits to Todd C. Miller and Paul de Weerd for helping me with it.
Comment 8 Augie Fackler 2019-05-20 19:16 UTC
I'd appreciate it if you could follow https://www.mercurial-scm.org/wiki/ContributingChanges (or at least attach an `hg export`ed commit to the bug) to make proper attribution feasible on my end.
Comment 9 Juan Francisco Cantero Hurtado 2019-05-21 13:38 UTC
Created attachment 2049 [details]
Fix compatibility problems with OpenBSD grep, take 3

Same patch but in the HG changeset patch format.
Comment 10 Augie Fackler 2019-05-21 14:07 UTC
Thanks, I've tweaked the summary a bit to make it a little shorter and added the (issue6122) suffix for the bug bot and pushed it. Many thanks!
Comment 11 HG Bot 2019-05-24 08:20 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/e0ac310bd033
Augie Fackler <augie@google.com>
tests: work around libressl being different about error strings (issue6122)

As far as I can tell, this is the right behavior. Thanks to Alex
Gaynor for checking what the string means by looking at libressl
sources for me.

Differential Revision: https://phab.mercurial-scm.org/D6410

(please test the fix)
Comment 12 HG Bot 2019-05-24 08:21 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/f22131315791
Juan Francisco Cantero Hurtado <iam@juanfra.info>
tests: make the grep pattern in remotefilelog-gcrepack portable (issue6122)

test-remotefilelog-gcrepack was using "\" to escape "|" in the grep pattern.
The most of implementations ignore "\" when it is followed by "|", so the regex
works. However, OpenBSD doesn't ignore "\" and considers "|" part of the text
instead of create two branches. Neither of both behaviors violate POSIX.

This change removes the unnecessary escape character and changes grep to egrep,
so the extended regular expression works on every unix.

This is part of the bug 6122. Tested on OpenBSD, GNU, FreeBSD, NetBSD, Solaris
11 and BusyBox.

Credits to Todd C. Miller, Paul de Weerd and Ingo Schwarze for helping me with
it.

(please test the fix)
Comment 13 Juan Francisco Cantero Hurtado 2019-05-24 19:24 UTC
Created attachment 2050 [details]
output of test-https

test-https is still failing.
Comment 14 Augie Fackler 2019-05-28 10:06 UTC
Ugh I missed that it's an abort. We'll probably need a way to make the test sniff out recent libressl versions and expect the different behavior. :(
Comment 15 Bugzilla 2019-10-26 00:03 UTC
Bug was inactive for 150 days, archiving