[PATCH RFC] sslutil: issue a warning when an "insecure" Python is used

Gregory Szorc gregory.szorc at gmail.com
Sun Mar 27 04:24:40 UTC 2016 

# HG changeset patch
# User Gregory Szorc <gregory.szorc at gmail.com>
# Date 1459052535 25200
#      Sat Mar 26 21:22:15 2016 -0700
# Node ID c9701ab2afa132dac924b2dfae30ab8371bbab97
# Parent  345f4fa4cc8912bb722ad3e35d68858487420bc6
sslutil: issue a warning when an "insecure" Python is used

Versions of CPython before 2.7.9 have insecure SSL/TLS. Before 2.7.9,
there is:

* No TLS 1.1 or 1.2
* No good, modern ciphersuites (because the good ones require TLS 1.1
  or 1.2)
* No SNI
* No system certificate access

Basically, it is using SSLv2 (broken), SSLv3 (broken), or TLS 1.0
(kinda/sorta broken but tolerated due to current popularity) along
with poor ciphersuites. It is vulnerable to POODLE. Basically, things
are marginally better than plain text. This is why web browsers and
other applications have disabled SSLv2 and SSLv3.

Version control tools - like web browsers - need to be trusted. Users
need to have confidence that connections to remote servers aren't
compromised.

Before this patch, Mercurial running on Python older than 2.7.9 will
print an ambiguous "certificate with fingerprint X not verified"
message when attempting a secure connection and the certificate can't
be validated (as signed by trusted CAs). Mercurial does perform
hostname verification (which is better than the Python standard
library on <2.7.9), however it doesn't have access to CA certs
(by default) to verify the cert. So there is no trust and therefore
little security.

After this patch, establishing a secure socket when the modern ssl
abilities provided by 2.7.9+ aren't available will result in an
additional warning saying the "connection may not be secure," which
is the truth because we support at most TLS 1.0 (and clients can
possibly be tricked into speaking SSLv2 or SSLv3).

The warning can be disabled via an undocumented config option. I
intend to document the option on the linked wiki page, but not in
the internal docs because config options that provide footguns
should be as undocumented as possible.

The changes to test-https.t are a bit more than I'd like. I had to
copy whole stanzas because the .t test parser doesn't understand
#ifdef..#else..#endif inside command output sections. If it did,
there would be much less duplication in the test and it would be
easier to comprehend and modify in the future.

I'm not thrilled about there now being 2 warnings in <2.7.9 (the
new warning and the fingerprint verification warning). I anticipate
bikeshedding.

diff --git a/mercurial/sslutil.py b/mercurial/sslutil.py
--- a/mercurial/sslutil.py
+++ b/mercurial/sslutil.py
@@ -55,16 +55,26 @@ try:
         # closed
         # - see http://bugs.python.org/issue13721
         if not sslsocket.cipher():
             raise error.Abort(_('ssl connection failed'))
         return sslsocket
 except AttributeError:
     def wrapsocket(sock, keyfile, certfile, ui, cert_reqs=ssl.CERT_NONE,
                    ca_certs=None, serverhostname=None):
+
+        # If we don't have a modern ssl module, issue a warning alerting
+        # the user about the problem.
+        # internal config: ui.warninsecuressl
+        if ui.configbool('ui', 'warninsecuressl', True):
+            ui.warn(_('(connection may not be secure because an old python '
+                      'version is being used; see '
+                       'https://www.mercurial-scm.org/wiki/InsecureSSL)\n'))
+            ui.setconfig('ui', 'warninsecuressl', False)
+
         sslsocket = ssl.wrap_socket(sock, keyfile, certfile,
                                     cert_reqs=cert_reqs, ca_certs=ca_certs,
                                     ssl_version=ssl.PROTOCOL_TLSv1)
         # check if wrap_socket failed silently because socket had been
         # closed
         # - see http://bugs.python.org/issue13721
         if not sslsocket.cipher():
             raise error.Abort(_('ssl connection failed'))
diff --git a/tests/test-https.t b/tests/test-https.t
--- a/tests/test-https.t
+++ b/tests/test-https.t
@@ -171,25 +171,39 @@ Apple's OpenSSL. This trick do not work 
   abort: error: *certificate verify failed* (glob)
   [255]
 
   $ DISABLEOSXDUMMYCERT="--config=web.cacerts=!"
 #endif
 
 clone via pull
 
+#if sslcontext
   $ hg clone https://localhost:$HGPORT/ copy-pull $DISABLEOSXDUMMYCERT
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   requesting all changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 4 changes to 4 files
   updating to branch default
   4 files updated, 0 files merged, 0 files removed, 0 files unresolved
+#else
+  $ hg clone https://localhost:$HGPORT/ copy-pull $DISABLEOSXDUMMYCERT
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 4 changes to 4 files
+  updating to branch default
+  4 files updated, 0 files merged, 0 files removed, 0 files unresolved
+#endif
+
   $ hg verify -R copy-pull
   checking changesets
   checking manifests
   crosschecking files in changesets and manifests
   checking files
   4 files, 1 changesets, 4 total revisions
   $ cd test
   $ echo bar > bar
@@ -197,129 +211,275 @@ clone via pull
   adding bar
   $ cd ..
 
 pull without cacert
 
   $ cd copy-pull
   $ echo '[hooks]' >> .hg/hgrc
   $ echo "changegroup = printenv.py changegroup" >> .hg/hgrc
+
+#if sslcontext
   $ hg pull $DISABLEOSXDUMMYCERT
   pulling from https://localhost:$HGPORT/
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   searching for changes
   adding changesets
   adding manifests
   adding file changes
   added 1 changesets with 1 changes to 1 files
   changegroup hook: HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_SOURCE=pull HG_TXNID=TXN:* HG_URL=https://localhost:$HGPORT/ (glob)
   (run 'hg update' to get a working copy)
+
+#else
+  $ hg pull $DISABLEOSXDUMMYCERT
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  searching for changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  changegroup hook: HG_NODE=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_NODE_LAST=5fed3813f7f5e1824344fdc9cf8f63bb662c292d HG_SOURCE=pull HG_TXNID=TXN:* HG_URL=https://localhost:$HGPORT/ (glob)
+  (run 'hg update' to get a working copy)
+#endif
+
   $ cd ..
 
 cacert configured in local repo
 
   $ cp copy-pull/.hg/hgrc copy-pull/.hg/hgrc.bu
   $ echo "[web]" >> copy-pull/.hg/hgrc
   $ echo "cacerts=`pwd`/pub.pem" >> copy-pull/.hg/hgrc
+
+#if sslcontext
   $ hg -R copy-pull pull --traceback
   pulling from https://localhost:$HGPORT/
   searching for changes
   no changes found
+#else
+  $ hg -R copy-pull pull --traceback
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  searching for changes
+  no changes found
+#endif
+
   $ mv copy-pull/.hg/hgrc.bu copy-pull/.hg/hgrc
 
 cacert configured globally, also testing expansion of environment
 variables in the filename
 
   $ echo "[web]" >> $HGRCPATH
   $ echo 'cacerts=$P/pub.pem' >> $HGRCPATH
+
+#if sslcontext
   $ P=`pwd` hg -R copy-pull pull
   pulling from https://localhost:$HGPORT/
   searching for changes
   no changes found
+#else
+  $ P=`pwd` hg -R copy-pull pull
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  searching for changes
+  no changes found
+#endif
+
+#if sslcontext
   $ P=`pwd` hg -R copy-pull pull --insecure
   pulling from https://localhost:$HGPORT/
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   searching for changes
   no changes found
+#else
+  $ P=`pwd` hg -R copy-pull pull --insecure
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  searching for changes
+  no changes found
+#endif
 
 cacert mismatch
 
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub.pem https://127.0.0.1:$HGPORT/
   pulling from https://127.0.0.1:$HGPORT/
   abort: 127.0.0.1 certificate error: certificate is for localhost
   (configure hostfingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca or use --insecure to connect insecurely)
   [255]
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub.pem https://127.0.0.1:$HGPORT/
+  pulling from https://127.0.0.1:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: 127.0.0.1 certificate error: certificate is for localhost
+  (configure hostfingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca or use --insecure to connect insecurely)
+  [255]
+#endif
+
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub.pem https://127.0.0.1:$HGPORT/ --insecure
   pulling from https://127.0.0.1:$HGPORT/
   warning: 127.0.0.1 certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   searching for changes
   no changes found
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub.pem https://127.0.0.1:$HGPORT/ --insecure
+  pulling from https://127.0.0.1:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: 127.0.0.1 certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  searching for changes
+  no changes found
+#endif
+
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub-other.pem
   pulling from https://localhost:$HGPORT/
   abort: error: *certificate verify failed* (glob)
   [255]
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub-other.pem
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: error: *certificate verify failed* (glob)
+  [255]
+#endif
+
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub-other.pem --insecure
   pulling from https://localhost:$HGPORT/
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   searching for changes
   no changes found
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub-other.pem --insecure
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  searching for changes
+  no changes found
+#endif
 
 Test server cert which isn't valid yet
 
   $ hg serve -R test -p $HGPORT1 -d --pid-file=hg1.pid --certificate=server-not-yet.pem
   $ cat hg1.pid >> $DAEMON_PIDS
+
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub-not-yet.pem https://localhost:$HGPORT1/
   pulling from https://localhost:$HGPORT1/
   abort: error: *certificate verify failed* (glob)
   [255]
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub-not-yet.pem https://localhost:$HGPORT1/
+  pulling from https://localhost:$HGPORT1/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: error: *certificate verify failed* (glob)
+  [255]
+#endif
 
 Test server cert which no longer is valid
 
   $ hg serve -R test -p $HGPORT2 -d --pid-file=hg2.pid --certificate=server-expired.pem
   $ cat hg2.pid >> $DAEMON_PIDS
+
+#if sslcontext
   $ hg -R copy-pull pull --config web.cacerts=pub-expired.pem https://localhost:$HGPORT2/
   pulling from https://localhost:$HGPORT2/
   abort: error: *certificate verify failed* (glob)
   [255]
+#else
+  $ hg -R copy-pull pull --config web.cacerts=pub-expired.pem https://localhost:$HGPORT2/
+  pulling from https://localhost:$HGPORT2/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: error: *certificate verify failed* (glob)
+  [255]
+#endif
 
 Fingerprints
 
   $ echo "[hostfingerprints]" >> copy-pull/.hg/hgrc
   $ echo "localhost = 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca" >> copy-pull/.hg/hgrc
   $ echo "127.0.0.1 = 914f1aff87249c09b6859b88b1906d30756491ca" >> copy-pull/.hg/hgrc
 
 - works without cacerts
+
+#if sslcontext
   $ hg -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
   5fed3813f7f5
+#else
+  $ hg -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  5fed3813f7f5
+#endif
 
 - multiple fingerprints specified and first matches
+
+#if sslcontext
   $ hg --config 'hostfingerprints.localhost=914f1aff87249c09b6859b88b1906d30756491ca, deadbeefdeadbeefdeadbeefdeadbeefdeadbeef' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
   5fed3813f7f5
+#else
+  $ hg --config 'hostfingerprints.localhost=914f1aff87249c09b6859b88b1906d30756491ca, deadbeefdeadbeefdeadbeefdeadbeefdeadbeef' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  5fed3813f7f5
+#endif
 
 - multiple fingerprints specified and last matches
+
+#if sslcontext
   $ hg --config 'hostfingerprints.localhost=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef, 914f1aff87249c09b6859b88b1906d30756491ca' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
   5fed3813f7f5
+#else
+  $ hg --config 'hostfingerprints.localhost=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef, 914f1aff87249c09b6859b88b1906d30756491ca' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  5fed3813f7f5
+#endif
 
 - multiple fingerprints specified and none match
 
+#if sslcontext
   $ hg --config 'hostfingerprints.localhost=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef, aeadbeefdeadbeefdeadbeefdeadbeefdeadbeef' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
   abort: certificate for localhost has unexpected fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca
   (check hostfingerprint configuration)
   [255]
+#else
+  $ hg --config 'hostfingerprints.localhost=deadbeefdeadbeefdeadbeefdeadbeefdeadbeef, aeadbeefdeadbeefdeadbeefdeadbeefdeadbeef' -R copy-pull id https://localhost:$HGPORT/ --config web.cacerts=!
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: certificate for localhost has unexpected fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca
+  (check hostfingerprint configuration)
+  [255]
+#endif
 
 - fails when cert doesn't match hostname (port is ignored)
+
+#if sslcontext
   $ hg -R copy-pull id https://localhost:$HGPORT1/
   abort: certificate for localhost has unexpected fingerprint 28:ff:71:bf:65:31:14:23:ad:62:92:b4:0e:31:99:18:fc:83:e3:9b
   (check hostfingerprint configuration)
   [255]
-
+#else
+  $ hg -R copy-pull id https://localhost:$HGPORT1/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: certificate for localhost has unexpected fingerprint 28:ff:71:bf:65:31:14:23:ad:62:92:b4:0e:31:99:18:fc:83:e3:9b
+  (check hostfingerprint configuration)
+  [255]
+#endif
 
 - ignores that certificate doesn't match hostname
+
+#if sslcontext
   $ hg -R copy-pull id https://127.0.0.1:$HGPORT/
   5fed3813f7f5
+#else
+  $ hg -R copy-pull id https://127.0.0.1:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  5fed3813f7f5
+#endif
 
 HGPORT1 is reused below for tinyproxy tests. Kill that server.
   $ killdaemons.py hg1.pid
 
 Prepare for connecting through proxy
 
   $ tinyproxy.py $HGPORT1 localhost >proxy.log </dev/null 2>&1 &
   $ while [ ! -f proxy.pid ]; do sleep 0; done
@@ -327,44 +487,86 @@ Prepare for connecting through proxy
 
   $ echo "[http_proxy]" >> copy-pull/.hg/hgrc
   $ echo "always=True" >> copy-pull/.hg/hgrc
   $ echo "[hostfingerprints]" >> copy-pull/.hg/hgrc
   $ echo "localhost =" >> copy-pull/.hg/hgrc
 
 Test unvalidated https through proxy
 
+#if sslcontext
   $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --insecure --traceback
   pulling from https://localhost:$HGPORT/
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
   searching for changes
   no changes found
+#else
+  $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --insecure --traceback
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostfingerprints or web.cacerts config setting)
+  searching for changes
+  no changes found
+#endif
 
 Test https with cacert and fingerprint through proxy
 
+#if sslcontext
   $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub.pem
   pulling from https://localhost:$HGPORT/
   searching for changes
   no changes found
+#else
+  $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub.pem
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  searching for changes
+  no changes found
+#endif
+
+#if sslcontext
   $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull https://127.0.0.1:$HGPORT/
   pulling from https://127.0.0.1:$HGPORT/
   searching for changes
   no changes found
+#else
+  $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull https://127.0.0.1:$HGPORT/
+  pulling from https://127.0.0.1:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  searching for changes
+  no changes found
+#endif
 
 Test https with cert problems through proxy
 
+#if sslcontext
   $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub-other.pem
   pulling from https://localhost:$HGPORT/
   abort: error: *certificate verify failed* (glob)
   [255]
+#else
+  $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub-other.pem
+  pulling from https://localhost:$HGPORT/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: error: *certificate verify failed* (glob)
+  [255]
+#endif
+
+#if sslcontext
   $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub-expired.pem https://localhost:$HGPORT2/
   pulling from https://localhost:$HGPORT2/
   abort: error: *certificate verify failed* (glob)
   [255]
-
+#else
+  $ http_proxy=http://localhost:$HGPORT1/ hg -R copy-pull pull --config web.cacerts=pub-expired.pem https://localhost:$HGPORT2/
+  pulling from https://localhost:$HGPORT2/
+  (connection may not be secure because an old python version is being used; see https://www.mercurial-scm.org/wiki/InsecureSSL)
+  abort: error: *certificate verify failed* (glob)
+  [255]
+#endif
 
   $ killdaemons.py hg0.pid
 
 #if sslcontext
 
 Start patched hgweb that requires client certificates:
 
   $ cat << EOT > reqclientcert.py

More information about the Mercurial-devel mailing list