[PATCH 5 of 6] upgrade: add a 'redeltafullall' mode
Boris Feld
boris.feld at octobus.net
Fri Dec 8 06:19:19 EST 2017
# HG changeset patch
# User Boris Feld <boris.feld at octobus.net>
# Date 1512674823 -3600
# Thu Dec 07 20:27:03 2017 +0100
# Node ID f020935650c9a3ba2c762524bde86243fbd39fe3
# Parent 5ac3ff23878f3a58a88540e776f9aa845ebbdd62
# EXP-Topic upgrade.config
# Available At https://bitbucket.org/octobus/mercurial-devel/
# hg pull https://bitbucket.org/octobus/mercurial-devel/ -r f020935650c9
upgrade: add a 'redeltafullall' mode
We add a new mode for delta recomputation, when selected, each full text will
go through the full "addrevision" mechanism again. This is slower than
"redeltaall" but this gives the opportunity for extensions to trigger special
logic. For example, the lfs extensions can decide to promote some revision to
lfs storage during the upgrade.
diff --git a/mercurial/revlog.py b/mercurial/revlog.py
--- a/mercurial/revlog.py
+++ b/mercurial/revlog.py
@@ -2264,7 +2264,9 @@ class revlog(object):
DELTAREUSESAMEREVS = 'samerevs'
DELTAREUSENEVER = 'never'
- DELTAREUSEALL = {'always', 'samerevs', 'never'}
+ DELTAREUSEFULLADD = 'fulladd'
+
+ DELTAREUSEALL = {'always', 'samerevs', 'never', 'fulladd'}
def clone(self, tr, destrevlog, addrevisioncb=None,
deltareuse=DELTAREUSESAMEREVS, aggressivemergedeltas=None):
@@ -2355,18 +2357,24 @@ class revlog(object):
if not cachedelta:
rawtext = self.revision(rev, raw=True)
- ifh = destrevlog.opener(destrevlog.indexfile, 'a+',
- checkambig=False)
- dfh = None
- if not destrevlog._inline:
- dfh = destrevlog.opener(destrevlog.datafile, 'a+')
- try:
- destrevlog._addrevision(node, rawtext, tr, linkrev, p1, p2,
- flags, cachedelta, ifh, dfh)
- finally:
- if dfh:
- dfh.close()
- ifh.close()
+
+ if deltareuse == self.DELTAREUSEFULLADD:
+ destrevlog.addrevision(rawtext, tr, linkrev, p1, p2,
+ cachedelta=cachedelta,
+ node=node, flags=flags)
+ else:
+ ifh = destrevlog.opener(destrevlog.indexfile, 'a+',
+ checkambig=False)
+ dfh = None
+ if not destrevlog._inline:
+ dfh = destrevlog.opener(destrevlog.datafile, 'a+')
+ try:
+ destrevlog._addrevision(node, rawtext, tr, linkrev, p1,
+ p2, flags, cachedelta, ifh, dfh)
+ finally:
+ if dfh:
+ dfh.close()
+ ifh.close()
if addrevisioncb:
addrevisioncb(self, rev, node)
diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
--- a/mercurial/upgrade.py
+++ b/mercurial/upgrade.py
@@ -369,6 +369,19 @@ def findoptimizations(repo):
'recomputed; this will likely drastically slow down '
'execution time')))
+ optimizations.append(improvement(
+ name='redeltafulladd',
+ type=optimisation,
+ description=_('every revision will be re-added as if it was new '
+ 'content. It will go through the full storage '
+ 'mechanism giving extensions a chance to process it '
+ '(eg. lfs). This is similar to "redeltaall" but even '
+ 'slower since more logic is involved.'),
+ upgrademessage=_('each revision will be added as new content to the '
+ 'internal storage; this will likely drastically slow '
+ 'down execution time, but some extensions might need '
+ 'it')))
+
return optimizations
def determineactions(repo, deficiencies, sourcereqs, destreqs):
@@ -618,6 +631,8 @@ def _upgraderepo(ui, srcrepo, dstrepo, r
deltareuse = revlog.revlog.DELTAREUSESAMEREVS
elif 'redeltamultibase' in actions:
deltareuse = revlog.revlog.DELTAREUSESAMEREVS
+ if 'redeltafulladd' in actions:
+ deltareuse = revlog.revlog.DELTAREUSEFULLADD
else:
deltareuse = revlog.revlog.DELTAREUSEALWAYS
diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
--- a/tests/test-upgrade-repo.t
+++ b/tests/test-upgrade-repo.t
@@ -100,6 +100,9 @@ An upgrade of a repository created with
redeltaall
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
+ redeltafulladd
+ every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "redeltaall" but even slower since more logic is involved.
+
--optimize can be used to add optimizations
@@ -121,6 +124,9 @@ An upgrade of a repository created with
redeltaall
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
+ redeltafulladd
+ every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "redeltaall" but even slower since more logic is involved.
+
Various sub-optimal detections work
@@ -196,6 +202,9 @@ Various sub-optimal detections work
redeltaall
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
+ redeltafulladd
+ every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "redeltaall" but even slower since more logic is involved.
+
$ hg --config format.dotencode=false debugupgraderepo
repository lacks features recommended by current config options:
@@ -235,6 +244,9 @@ Various sub-optimal detections work
redeltaall
deltas within internal storage will always be recalculated without reusing prior deltas; this will likely make execution run several times slower; this optimization is typically not needed
+ redeltafulladd
+ every revision will be re-added as if it was new content. It will go through the full storage mechanism giving extensions a chance to process it (eg. lfs). This is similar to "redeltaall" but even slower since more logic is involved.
+
$ cd ..
@@ -406,6 +418,40 @@ store files with special filenames aren'
removing temporary repository $TESTTMP/store-filenames/.hg/upgrade.* (glob)
copy of old repository backed up at $TESTTMP/store-filenames/.hg/upgradebackup.* (glob)
the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
+ $ hg debugupgraderepo --run --optimize redeltafulladd
+ upgrade will perform the following actions:
+
+ requirements
+ preserved: dotencode, fncache, generaldelta, revlogv1, store
+
+ redeltafulladd
+ each revision will be added as new content to the internal storage; this will likely drastically slow down execution time, but some extensions might need it
+
+ beginning upgrade...
+ repository locked and read-only
+ creating temporary repository to stage migrated data: $TESTTMP/store-filenames/.hg/upgrade.* (glob)
+ (it is safe to interrupt this process any time before data migration completes)
+ migrating 3 total revisions (1 in filelogs, 1 in manifests, 1 in changelog)
+ migrating 109 bytes in store; 107 bytes tracked data
+ migrating 1 filelogs containing 1 revisions (0 bytes in store; 0 bytes tracked data)
+ finished migrating 1 filelog revisions across 1 filelogs; change in size: 0 bytes
+ migrating 1 manifests containing 1 revisions (46 bytes in store; 45 bytes tracked data)
+ finished migrating 1 manifest revisions across 1 manifests; change in size: 0 bytes
+ migrating changelog containing 1 revisions (63 bytes in store; 62 bytes tracked data)
+ finished migrating 1 changelog revisions; change in size: 0 bytes
+ finished migrating 3 total revisions; total change in store size: 0 bytes
+ copying .XX_special_filename
+ copying phaseroots
+ data fully migrated to temporary repository
+ marking source repository as being upgraded; clients will be unable to read from repository
+ starting in-place swap of repository data
+ replaced files will be backed up at $TESTTMP/store-filenames/.hg/upgradebackup.* (glob)
+ replacing store...
+ store replacement complete; repository was inconsistent for 0.0s
+ finalizing requirements file and making repository readable again
+ removing temporary repository $TESTTMP/store-filenames/.hg/upgrade.* (glob)
+ copy of old repository backed up at $TESTTMP/store-filenames/.hg/upgradebackup.* (glob)
+ the old repository will not be deleted; remove it to free up disk space once the upgraded repository is verified
$ cd ..
More information about the Mercurial-devel
mailing list