[PATCH 07 of 10] repair: begin implementation of in-place upgrading

Gregory Szorc gregory.szorc at gmail.com
Sun Nov 6 00:40:23 EDT 2016


# HG changeset patch
# User Gregory Szorc <gregory.szorc at gmail.com>
# Date 1478393266 25200
#      Sat Nov 05 17:47:46 2016 -0700
# Node ID 958bcf2577608bbb6d8ae078cde0ca451f3ab31a
# Parent  82799afa72be5bb540b2b07cd878f307622c4354
repair: begin implementation of in-place upgrading

Now that all the upgrade planning work is complete, we can start
doing the real work: actually upgrading a repository.

The main goal of this patch is to get the "framework" for running
in-place upgrade actions in place.

Rather than get too clever and low-level with regards to in-place
upgrades, our strategy for an in-place upgrade is to create a new,
temporary repository, copy data to it, then replace the old data
with the new. This allows us to reuse a lot of code in localrepo.py
around store interaction, which will eventually consume the bulk of
the upgrade code.

But we have to start small. This patch implements adding new
repository requirements. But it still sets up a temporary
repository and locks it and the source repo before performing the
requirements file swap. This means all the plumbing is in place
to implement store copying in subsequent patches.

diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -10,6 +10,7 @@ from __future__ import absolute_import
 
 import errno
 import hashlib
+import tempfile
 
 from .i18n import _
 from .node import short
@@ -19,6 +20,7 @@ from . import (
     error,
     exchange,
     obsolete,
+    scmutil,
     util,
 )
 
@@ -535,8 +537,41 @@ def upgradesummarizeactions(repo, action
 
     return l, handled
 
+def _upgraderepo(ui, srcrepo, dstrepo, requirements):
+    """Do the low-level work of upgrading a repository.
+
+    The upgrade is effectively performed as a copy between a source
+    repository and a temporary destination repository.
+
+    The resource repository is intended to be read only for as long as
+    possible so the upgrade can abort at any time without corrupting
+    the source.
+    """
+    assert srcrepo.currentwlock()
+    assert dstrepo.currentwlock()
+
+    # TODO copy store
+
+    ui.write(_('starting in-place swap of repository data\n'))
+    ui.warn(_('(clients may error or see inconsistent repository data until '
+              'this operation completes)\n'))
+
+    backuppath = tempfile.mkdtemp(prefix='upgradebackup.', dir=srcrepo.path)
+    backupvfs = scmutil.vfs(backuppath)
+    ui.write(_('replaced files will be backed up at %s\n') %
+             backuppath)
+
+    # We first write the requirements file. Any new requirements will lock
+    # out legacy clients.
+    ui.write(_('updating requirements in %s\n') % srcrepo.join('requires'))
+    util.copyfile(srcrepo.join('requires'), backupvfs.join('requires'))
+    scmutil.writerequires(srcrepo.vfs, requirements)
+
 def upgraderepo(ui, repo, dryrun=False):
     """Upgrade a repository in place."""
+    # Avoid cycle: cmdutil -> repair -> localrepo -> cmdutil
+    from . import localrepo
+
     repo = repo.unfiltered()
     deficiencies, actions = upgradefinddeficiencies(repo)
 
@@ -583,3 +618,27 @@ def upgraderepo(ui, repo, dryrun=False):
     if dryrun:
         ui.warn(_('dry run requested; not continuing with upgrade\n'))
         return 1
+
+    ui.warn(_('starting repository upgrade\n'))
+
+    # Now we're finally at the actual upgrade part.
+    with repo.wlock():
+        with repo.lock():
+            ui.write(_('source repository locked and read-only\n'))
+            # It is easier to create a new repo than to instantiate all the
+            # components (like the store) separately.
+            tmppath = tempfile.mkdtemp(prefix='upgrade.', dir=repo.path)
+            try:
+                ui.write(_('creating temporary repository to stage migrated '
+                           'data: %s\n') % tmppath)
+                dstrepo = localrepo.localrepository(repo.baseui,
+                                                    path=tmppath,
+                                                    create=True)
+
+                with dstrepo.wlock():
+                    with dstrepo.lock():
+                        _upgraderepo(ui, repo, dstrepo, newreqs)
+
+            finally:
+                ui.write(_('removing temporary repository %s\n') % tmppath)
+                repo.vfs.rmtree(tmppath, forcibly=True)
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -6,6 +6,14 @@
   checking repository requirements...
   preserving repository requirements: dotencode, fncache, generaldelta, revlogv1, store
   no notable upgrade changes identified
+  starting repository upgrade
+  source repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/empty/.hg/upgrade.* (glob)
+  starting in-place swap of repository data
+  (clients may error or see inconsistent repository data until this operation completes)
+  replaced files will be backed up at $TESTTMP/empty/.hg/upgradebackup.* (glob)
+  updating requirements in $TESTTMP/empty/.hg/requires
+  removing temporary repository $TESTTMP/empty/.hg/upgrade.* (glob)
 
 dry run works
 
@@ -40,6 +48,14 @@ Various sub-optimal detections work
   * dotencode repository layout will be used; the repository will be able be more resilient to storing paths beginning with periods or spaces
   * fncache repository layout will be used; the repository will be more resilient storing long paths and special filenames
   * repository data will be re-encoded as "generaldelta"; files inside the repository should be smaller, read times should decrease, and interacting with other generaldelta repositories should be faster
+  starting repository upgrade
+  source repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/empty/.hg/upgrade.* (glob)
+  starting in-place swap of repository data
+  (clients may error or see inconsistent repository data until this operation completes)
+  replaced files will be backed up at $TESTTMP/empty/.hg/upgradebackup.* (glob)
+  updating requirements in $TESTTMP/empty/.hg/requires
+  removing temporary repository $TESTTMP/empty/.hg/upgrade.* (glob)
 
   $ cd ..
 
@@ -90,3 +106,54 @@ Cannot add specific requirements during 
   checking repository requirements...
   abort: cannot upgrade repository; proposed new requirement cannot be added: treemanifest
   [255]
+
+Upgrading a repository to generaldelta works
+
+  $ hg --config format.usegeneraldelta=false init upgradegd
+  $ cd upgradegd
+  $ touch f0
+  $ hg -q commit -A -m initial
+  $ touch f1
+  $ hg -q commit -A -m 'add f1'
+  $ hg -q up -r 0
+  $ touch f2
+  $ hg -q commit -A -m 'add f2'
+
+  $ hg debugupgraderepo
+  the following deficiencies with the existing repository have been identified:
+  
+  * not using generaldelta storage; repository is larger and slower than it could be, pulling from generaldelta repositories will be slow
+  
+  checking repository requirements...
+  preserving repository requirements: dotencode, fncache, revlogv1, store
+  adding repository requirements: generaldelta
+  upgrading the repository will make the following significant changes:
+  
+  * repository data will be re-encoded as "generaldelta"; files inside the repository should be smaller, read times should decrease, and interacting with other generaldelta repositories should be faster
+  starting repository upgrade
+  source repository locked and read-only
+  creating temporary repository to stage migrated data: $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+  starting in-place swap of repository data
+  (clients may error or see inconsistent repository data until this operation completes)
+  replaced files will be backed up at $TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
+  updating requirements in $TESTTMP/upgradegd/.hg/requires
+  removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
+
+Original requirements backed up
+
+  $ cat .hg/upgradebackup.*/requires
+  dotencode
+  fncache
+  revlogv1
+  store
+
+generaldelta added to original requirements files
+
+  $ cat .hg/requires
+  dotencode
+  fncache
+  generaldelta
+  revlogv1
+  store
+
+  $ cd ..


More information about the Mercurial-devel mailing list