D5411: sqlitestore: create new connections on new PIDs

indygreg (Gregory Szorc) phabricator at mercurial-scm.org
Wed Dec 12 00:00:53 UTC 2018


indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  If the Mercurial process fork()s, the Python thread ID remains
  unchanged. The previous code for returning a SQLite connection would
  recycle the existing connection among all children.
  
  Mercurial can fork() when performing working directory updates.
  For reasons I don't fully understand, the recycling of even a
  read-only SQLite connection was resulting in Python raising a
  "DatabaseError: database disk image is malformed" exception. This
  message comes from the bowels of SQLite. I suspect there is some
  internal client state in the SQLite database somewhere and having
  multiple clients race to update it results in badness. Who knows.
  
  This commit teaches the "get a SQLite connection" logic to also
  verify the PID matches before returning an existing connection.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5411

AFFECTED FILES
  hgext/sqlitestore.py

CHANGE DETAILS

diff --git a/hgext/sqlitestore.py b/hgext/sqlitestore.py
--- a/hgext/sqlitestore.py
+++ b/hgext/sqlitestore.py
@@ -76,6 +76,7 @@
 )
 from mercurial.utils import (
     interfaceutil,
+    procutil,
     storageutil,
 )
 
@@ -1005,17 +1006,18 @@
 
     @property
     def _dbconn(self):
-        # SQLite connections can only be used on the thread that created
-        # them. In most cases, this "just works." However, hgweb uses
-        # multiple threads.
-        tid = threading.current_thread().ident
+        # SQLite connections can only be used on the OS and Python thread that
+        # created them. In most cases, this "just works." However, hgweb uses
+        # multiple Python threads. And Mercurial may fork. So we need to check
+        # global state before returning an existing connection.
+        key = (procutil.getpid(), threading.current_thread().ident)
 
         if self._db:
-            if self._db[0] == tid:
+            if self._db[0] == key:
                 return self._db[1]
 
         db = makedb(self.svfs.join('db.sqlite'))
-        self._db = (tid, db)
+        self._db = (key, db)
 
         return db
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel


More information about the Mercurial-devel mailing list