Opposite of --insecure is what?

Sat Feb 5 20:37:51 CST 2011

On Feb 6, 2011, at 2:48 PM, Steve Borho wrote:

> On Sat, Feb 5, 2011 at 4:12 PM, Jason Harris <jason at jasonfharris.com> wrote:
>> 
>> On Feb 5, 2011, at 12:26 AM, Mads Kiilerich wrote:
>> 
>>> On 02/04/2011 10:09 AM, Jason Harris wrote:
>>>> Ok in making MacHg work with certificates for https:// and fingerprints and
>>>> stuff the following issue has come up... Ironically I now have almost the
>>>> opposite question to the ones I previously asked. How do I make the operation
>>>> always fail if there is no verifiable certificate or fingerprint?
>>> 
>>> For now the way to do that is to specify a web.cacerts file. It can't be empty, but you can generate a random self-signed certificate and throw away the key.
>>> 
>>> It would make sense if Mercurial by default aborted instead of issuing a warning (unless --insecure was specified). Such a change would in my opinion have been too invasive for a stable bugfix release - that is one of the reasons we just give a warning (or a number of them ...).
>>> 
>>> The packaging guideline has however already been changed to recommend shipping Mercurial with web.cacerts configured and the windows installers already does that. That effectively changed the default.
>> 
>> Ok that makes sense. The wiki page wasn't entirely clear which bunch of certificates was recommended. In the end I used the ones here:
>> 
>> http://curl.haxx.se/docs/caextract.html
>> 
>> Or is there a better recommendation?
>> 
>> 
>>> Developers: Should we change the insecure warning to an abort in 1.8? With fallback to insecure+warning if web.cacerts=! ?
>>> 
>>>> But if I set up web.cacerts then things fail outright when connecting to a
>>>> server with a self-signed certificate, but no fingerprint info is printed. Eg
>>>> 
>>>> [Bolt:/Development/MacHgDev/MacHgDH] MacHgDH 2211(2211) ⌘ hg pull https://***@jasonfharris.com/hg/MacHg
>>>> abort: error: _ssl.c:480: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
>>>> 
>>>> (Note this is a scary error and really what has happened is that the self-signed
>>>> certificate was unknown to Mercurial. It wasn't that it was trying to spoof
>>>> something... The message shouldn't cry wolf. Maybe the message should be a
>>>> little lighter for this case? (I gather from the code that this is coming
>>>> through from a layer below Mercurial...)
>>> 
>>> Yes, that message is all we get from Python. AFAIK there is no way we reliably can give better error messages, and there is no way we can retrieve a fingerprint from the failing https session.
>>> 
>>> Mercurial _could_ create another connection just to retrieve the fingerprint. Creating extra network connections in a fatal exception handler is however not necessarily a good ide. Instead you can do it by connecting again, either insecurely or with an invalid fingerprint - or use a python one-liner.
>> 
>> 
>> Actually what is your one-liner? After googling around a bit I got down to basically the shell script:
>> 
>> #!/bin/sh
>> echo | openssl s_client -connect $1:$2 | openssl x509 -fingerprint -noout -sha1 2>&1
>> 
>> But if I call this script say getHTTPSfingerprint.sh then the results are something like:
>> 
>> [Bolt:~] $  getHTTPSfingerprint.sh jasonfharris.com 443
>> depth=0 /C=NZ/ST=*****/L=*****/O=Jason Harris/OU=Internet Security Division/CN=jasonfharris.com/emailAddress=*****@gmail.com
>> verify error:num=18:self signed certificate
>> verify return:1
>> depth=0 /C=NZ/ST=*****/L=*****/O=Jason Harris/OU=Internet Security Division/CN=jasonfharris.com/emailAddress=j*****@gmail.com
>> verify return:1
>> DONE
>> SHA1 Fingerprint=B8:03:9F:99:31:F9:5E:4B:AB:5B:7E:5F:AF:FF:E1:EC:F6:F0:2E:55
>> [Bolt:~] $
>> 
>> I can of course pick out the end key with a regex but its starting to then get a bit ugly... you might have a nicer way to do this then?
>> 
>> 
>> 
>>>> Why do I need a --authentication strict option?
>>> ...
>>> 
>>> Some brief comments:
>>> 
>>> "authentication" usually refers to the user authenticating to the server, and Mercurial already has an "auth" section for that. This should be called something else - I don't know what. Perhaps something like "serveridentity".
>> 
>> Yes. You are right. I was floundering round for a good name for this as well.
>> 
>> --serveridentity secure
>> --serveridentity flexible
>> --serveridentity insecure
>> --serveridentity ignore
>> 
>> Of these, in GUI clients I would imagine that the two extreme cases are really used. Ie totally secure and totally insecure with no warnings.
>> 
>> 
>>> We would certainly like to see much of what you are asking for.
>>> 
>>> Mercurial is however constrained by what Python ssl allows.
>>> 
>>> Another constraint is backward compatibility. Some of our users start complaining whenever anything changes.
>>> 
>>> Mercurial would certainly look differently in this area if we could redesign it from scratch without these constraints.
>> 
>> Thanks for your comments... :)  It makes the situation more "understandable".
>> 
>>> TortoiseHg 1.9 has the same challenges as you have, and Steve has been through the same thoughts a month ago and worked closely with the other Mercurial developers to create an optimal solution. Check it out and get inspiration there.
>> 
>> Steve, any pointers to where I should look...
> 
> TortoiseHg 1.9.2 has a 'security' dialog that can be opened when an
> https URL is selected.
> 
> https://bitbucket.org/tortoisehg/thg/src/7d1ab6d4151a/tortoisehg/hgqt/sync.py#cl-950
> 
> The dialog has two halves.  The top half is labeled 'Secure HTTPS
> Connection' and has set of radio buttons that select CA certs, a
> fingerprint, or --insecure.  The bottom half is labeled "User
> Authentication" and provides a very simplified interface to the [auth]
> section.
> 
> If the user selects the fingerprint option, they can query the
> fingerprint from the host or they can paste a fingerprint from the
> output log.  To support a per-host --insecure option, we define an
> [insecurehosts] configuration section.  If the hostname is in that
> section and has a "True" boolean value we add --insecure to all
> command lines that talk to the host over https.

Ahh... Thanks for the explanation!!

I sort of understand this, but in permanent setup, whats the difference between just marking something --insecure in all traffic and just storing its host key which you can fetch automatically?

I wanted the --insecure option so I could do discovery and other operations without worrying about security, and then only later actually worrying about security. But to a large extent storing the fingerprint which comes from the host and --insecure is the same, right?

> I've uploaded a snapshot of the dialog on Mac:
> 
> https://bitbucket.org/tortoisehg/thg/downloads/security.png

How do you do the automatic detection and switch between real errors that can be detected if a spoofed certificate and a self signed certificate. I wouldn't want to get a spoofing error and still recommend to the user, the host could not be verified but has the following fingerprint would you like to use this fingerprint?

Thanks,
  Jas