[PATCH 3 of 3] [RFC] sslutil: try to find and use system CA file when appropriate

[Gregory Szorc]

On Wed, Jun 29, 2016 at 8:14 PM, Gregory Szorc <gregory.szorc at gmail.com>
wrote:

> On Sat, Jun 25, 2016 at 7:06 PM, Pierre-Yves David <
> pierre-yves.david at ens-lyon.org> wrote:
>
>>
>>
>> On 06/25/2016 06:37 PM, Gregory Szorc wrote:
>> > # HG changeset patch
>> > # User Gregory Szorc <gregory.szorc at gmail.com>
>> > # Date 1466866076 25200
>> > #      Sat Jun 25 07:47:56 2016 -0700
>> > # Node ID fb5033513e39b96777f9794b9542c3632a64fa75
>> > # Parent  13818ab440e16c575b09a6e4583f9a17550a6e52
>> > [RFC] sslutil: try to find and use system CA file when appropriate
>>
>> I like the idea of trying to reduce the pain of user still on 2.6 by
>> looking the CA semi-automatically. However, I agree with you that doing
>> it might be suboptimal (especially security wise), I do not have a
>> strong opinion about doing it or not in absolute yet, but we should
>> probably at least issue a warning when doing so. With a pointer to why
>> the warning exist and how to suppress it.
>> As I understand it, in the same situation before we would have accept
>> the certificate with a warning so this seems like a proper evolution of
>> the behavior.
>
>
> So... TIL that distro packagers don't care about Python security either.
>
> On Debian Jessie and Ubuntu 16.04:
>
> $ /usr/bin/python2.7
> >>> import ssl
> >>> ctx = ssl.create_default_context()
> >>> ctx.load_default_certs()
> >>> ctx.cert_store_stats()
> {'x509': 0, 'x509_ca': 0, 'crl': 0}
> >>> ctx.load_verify_locations(cafile='/etc/ssl/certs/ca-certificates.crt')
> >>> ctx.cert_store_stats()
> {'x509': 173, 'x509_ca': 173, 'crl': 0}
>
> The whole point of ssl.SSLContext.load_default_certs() is it is supposed
> to load the default/system CA certificates from default locations. As you
> can see, this flat out doesn't work on Debian and Ubuntu 16.04. The result
> is that /usr/bin/python2.7 on these distros is unable to load any CA
> certificates.
>
> Some very brief perusing of the Internets recommends using non-stdlib
> modules [which know the locations of system CA certs on various
> distributions] to load CA certs. Seriously.
>

The situation on Debian and Ubuntu doesn't appear to be as bad as initially
thought.

ssl.SSLContext.load_verify_locations() specifies 2 ways to load certs: via
a single file and via a path containing files with hashes of CAs.

When you load a directory, the certs are loaded on demand as they are
encountered through connection activity. Only then do the certs show up in
cert_store_stats().

Debian and Ubuntu appear to be using this directory-based cert loading.
RHEL is using the file-based one. And FWIW Mercurial only supports
specifying a config option for loading certs from a file. We /could/
support loading from a directory if we wanted. Although I haven't heard
anybody clamor for this, so I'm not going to do it.

Anyway, what this means is that "are any CAs loaded" is a bit difficult to
implement on systems loading certs from directories because CA count == 0
could mean either no CAs loaded or the CA wanted by the server connection
just isn't present locally. I'll likely have to tweak the warning message
for "no CAs loaded" that I added a few hours ago and Yuya recently queued.
I'll also have to look into behavior on Windows and OS X to see if they are
reporting counts in cert_store_stats(). Perhaps it would be best to prune
that last patch from the committed repo until I get a handle on things. If
not, I'll send follow-up commits.


>
> Fortunately, RHEL 7 and CentOS 7 have a better outcome:
>
> $ /usr/bin/python2.7
> >>> import ssl
> >>> ctx = ssl.create_default_context()
> >>> ctx.load_default_certs()
> >>> ctx.cert_store_stats()
> {'x509': 167, 'x509_ca': 167, 'crl': 0}
>
> Kudos to RedHat for configuring their Python package to load the system CA
> certificates and provide working CA verification when using the standard
> library.
>
> Anyway, given the popularity of Debian-based distros, I think we'll need
> to do something to detect the system CA certs file
> (/etc/ssl/certs/ca-certificates.crt) since Debian's Python package isn't
> willing to load it for us. Expect a v2 of this RFC patch from me eventually.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20160630/48138b5e/attachment.html>