[PATCH 6 of 8] upgrade: add an argument to control manifest upgrade

Gregory Szorc gregory.szorc at gmail.com
Mon Aug 5 22:44:51 EDT 2019


On Mon, Aug 5, 2019 at 10:04 AM Pierre-Yves David <
pierre-yves.david at ens-lyon.org> wrote:

> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david at octobus.net>
> # Date 1564439752 -7200
> #      Tue Jul 30 00:35:52 2019 +0200
> # Node ID 909da55a9517693f2911793272e03cb7f108cfaf
> # Parent  5b8a0d3b7596be28a31faac4ffc499da46c0b62e
> # EXP-Topic upgrade-select
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 909da55a9517
> upgrade: add an argument to control manifest upgrade
>

Before we land this and the next commit, I'd like you to think about:

1) Filelog support. If we have a flag for manifests and changelogs,
shouldn't there be one for files?
2) How the presence of multiple flags interacts. Assuming we add a flag to
control files, what happens when you supply --manifest --changelog or
--no-manifest --no-changelog? Does that behavior make sense?
3) How do we extend this selection to other storage primitives? Do we need
to keep adding arguments for each storage primitive type?

I haven't thought this one through, but I have a suspicion that once
someone does, we'll decide that having arguments named after actions and
taking nouns will be the most scalable solution. e.g. --include manifest
--exclude changeset --exclude files. (If you go this route please do not
use include/exclude as we may want to reserve those for specifying a
fileset of files to upgrade, like many other commands.)


>
> The argument can be used to only "clone" manifest revlog or clone all of
> them
> but this one. The selection will make more sense once we have a
> `--changelog`
> flag in the next changesets.
>
> diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
> --- a/mercurial/debugcommands.py
> +++ b/mercurial/debugcommands.py
> @@ -2848,8 +2848,9 @@ def debugupdatecaches(ui, repo, *pats, *
>      ('o', 'optimize', [], _('extra optimization to perform'), _('NAME')),
>      ('', 'run', False, _('performs an upgrade')),
>      ('', 'backup', True, _('keep the old repository content around')),
> +    ('', 'manifest', None, _('select the manifest for upgrade')),
>  ])
> -def debugupgraderepo(ui, repo, run=False, optimize=None, backup=True):
> +def debugupgraderepo(ui, repo, run=False, optimize=None, backup=True,
> **opts):
>      """upgrade a repository to use different features
>
>      If no arguments are specified, the repository is evaluated for upgrade
> @@ -2867,9 +2868,15 @@ def debugupgraderepo(ui, repo, run=False
>      rename some directories inside the ``.hg`` directory. On most
> machines, this
>      should complete almost instantaneously and the chances of a consumer
> being
>      unable to access the repository should be low.
> +
> +    By default, all revlog will be upgraded. You can restrict this using
> flag
> +    such as `--manifest`:
> +
> +      * `--manifest`: only optimize the manifest
> +      * `--no-manifest`: optimize all revlog but the manifest
>      """
>      return upgrade.upgraderepo(ui, repo, run=run, optimize=optimize,
> -                               backup=backup)
> +                               backup=backup, **opts)
>
>  @command('debugwalk', cmdutil.walkopts, _('[OPTION]... [FILE]...'),
>           inferrepo=True)
> diff --git a/mercurial/upgrade.py b/mercurial/upgrade.py
> --- a/mercurial/upgrade.py
> +++ b/mercurial/upgrade.py
> @@ -866,13 +866,32 @@ def _upgraderepo(ui, srcrepo, dstrepo, r
>
>      return backuppath
>
> -def upgraderepo(ui, repo, run=False, optimize=None, backup=True):
> +def upgraderepo(ui, repo, run=False, optimize=None, backup=True,
> +                manifest=None):
>      """Upgrade a repository in place."""
>      if optimize is None:
>          optimize = []
>      optimize = set(legacy_opts_map.get(o, o) for o in optimize)
>      repo = repo.unfiltered()
>
> +    revlogs = set(UPGRADE_ALL_REVLOGS)
> +    specentries = (('m', manifest),)
> +    specified = [(y, x) for (y, x) in specentries if x is not None]
> +    if specified:
> +        # we have some limitation on revlogs to be recloned
> +        if any(x for y, x in specified):
> +            revlogs = set()
> +            for r, enabled in specified:
> +                if enabled:
> +                    if r == 'm':
> +                        revlogs.add(UPGRADE_MANIFEST)
> +        else:
> +            # none are enabled
> +            for r, __ in specified:
> +                if r == 'm':
> +                    revlogs.discard(UPGRADE_MANIFEST)
> +
> +
>      # Ensure the repository can be upgraded.
>      missingreqs = requiredsourcerequirements(repo) - repo.requirements
>      if missingreqs:
> @@ -1022,7 +1041,7 @@ def upgraderepo(ui, repo, run=False, opt
>
>              with dstrepo.wlock(), dstrepo.lock():
>                  backuppath = _upgraderepo(ui, repo, dstrepo, newreqs,
> -                                          upgradeactions)
> +                                          upgradeactions, revlogs=revlogs)
>              if not (backup or backuppath is None):
>                  ui.write(_('removing old repository content%s\n') %
> backuppath)
>                  repo.vfs.rmtree(backuppath, forcibly=True)
> diff --git a/tests/test-completion.t b/tests/test-completion.t
> --- a/tests/test-completion.t
> +++ b/tests/test-completion.t
> @@ -312,7 +312,7 @@ Show all commands + options
>    debuguigetpass: prompt
>    debuguiprompt: prompt
>    debugupdatecaches:
> -  debugupgraderepo: optimize, run, backup
> +  debugupgraderepo: optimize, run, backup, manifest
>    debugwalk: include, exclude
>    debugwhyunstable:
>    debugwireargs: three, four, five, ssh, remotecmd, insecure
> diff --git a/tests/test-upgrade-repo.t b/tests/test-upgrade-repo.t
> --- a/tests/test-upgrade-repo.t
> +++ b/tests/test-upgrade-repo.t
> @@ -518,9 +518,126 @@ unless --no-backup is passed
>    removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
>    $ ls -1 .hg/ | grep upgradebackup
>    [1]
> +
> +We can restrict optimization to some revlog:
> +
> +  $ hg debugupgrade --optimize re-delta-parent --run --manifest
> --no-backup --debug --traceback
> +  upgrade will perform the following actions:
> +
> +  requirements
> +     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog,
> store
> +
> +  re-delta-parent
> +     deltas within internal storage will choose a new base revision if
> needed
> +
> +  beginning upgrade...
> +  repository locked and read-only
> +  creating temporary repository to stage migrated data:
> $TESTTMP/upgradegd/.hg/upgrade.* (glob)
> +  (it is safe to interrupt this process any time before data migration
> completes)
> +  migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in
> changelog)
> +  migrating 917 bytes in store; 401 bytes tracked data
> +  migrating 3 filelogs containing 3 revisions (192 bytes in store; 0
> bytes tracked data)
> +  blindly copying data/f0.i containing 1 revisions
> +  blindly copying data/f1.i containing 1 revisions
> +  blindly copying data/f2.i containing 1 revisions
> +  finished migrating 3 filelog revisions across 3 filelogs; change in
> size: 0 bytes
> +  migrating 1 manifests containing 3 revisions (349 bytes in store; 220
> bytes tracked data)
> +  cloning 3 revisions from 00manifest.i
> +  finished migrating 3 manifest revisions across 1 manifests; change in
> size: 0 bytes
> +  migrating changelog containing 3 revisions (376 bytes in store; 181
> bytes tracked data)
> +  blindly copying 00changelog.i containing 3 revisions
> +  finished migrating 3 changelog revisions; change in size: 0 bytes
> +  finished migrating 9 total revisions; total change in store size: 0
> bytes
> +  copying phaseroots
> +  data fully migrated to temporary repository
> +  marking source repository as being upgraded; clients will be unable to
> read from repository
> +  starting in-place swap of repository data
> +  replaced files will be backed up at
> $TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
> +  replacing store...
> +  store replacement complete; repository was inconsistent for *s (glob)
> +  finalizing requirements file and making repository readable again
> +  removing old repository content$TESTTMP/upgradegd/.hg/upgradebackup.*
> (glob)
> +  removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
> +
> +Check that the repo still works fine
> +
> +  $ hg log -G --patch
> +  @  changeset:   2:b5a3b78015e5
> +  |  tag:         tip
> +  |  parent:      0:ba592bf28da2
> +  |  user:        test
> +  |  date:        Thu Jan 01 00:00:00 1970 +0000
> +  |  summary:     add f2
> +  |
> +  |
> +  | o  changeset:   1:da8c0fc4833c
> +  |/   user:        test
> +  |    date:        Thu Jan 01 00:00:00 1970 +0000
> +  |    summary:     add f1
> +  |
> +  |
> +  o  changeset:   0:ba592bf28da2
> +     user:        test
> +     date:        Thu Jan 01 00:00:00 1970 +0000
> +     summary:     initial
> +
> +
> +
> +  $ hg verify
> +  checking changesets
> +  checking manifests
> +  crosschecking files in changesets and manifests
> +  checking files
> +  checked 3 changesets with 3 changes to 3 files
> +
> +Check we can select negatively
> +
> +  $ hg debugupgrade --optimize re-delta-parent --run --no-manifest
> --no-backup --debug --traceback
> +  upgrade will perform the following actions:
> +
> +  requirements
> +     preserved: dotencode, fncache, generaldelta, revlogv1, sparserevlog,
> store
> +
> +  re-delta-parent
> +     deltas within internal storage will choose a new base revision if
> needed
> +
> +  beginning upgrade...
> +  repository locked and read-only
> +  creating temporary repository to stage migrated data:
> $TESTTMP/upgradegd/.hg/upgrade.* (glob)
> +  (it is safe to interrupt this process any time before data migration
> completes)
> +  migrating 9 total revisions (3 in filelogs, 3 in manifests, 3 in
> changelog)
> +  migrating 917 bytes in store; 401 bytes tracked data
> +  migrating 3 filelogs containing 3 revisions (192 bytes in store; 0
> bytes tracked data)
> +  cloning 1 revisions from data/f0.i
> +  cloning 1 revisions from data/f1.i
> +  cloning 1 revisions from data/f2.i
> +  finished migrating 3 filelog revisions across 3 filelogs; change in
> size: 0 bytes
> +  migrating 1 manifests containing 3 revisions (349 bytes in store; 220
> bytes tracked data)
> +  blindly copying 00manifest.i containing 3 revisions
> +  finished migrating 3 manifest revisions across 1 manifests; change in
> size: 0 bytes
> +  migrating changelog containing 3 revisions (376 bytes in store; 181
> bytes tracked data)
> +  cloning 3 revisions from 00changelog.i
> +  finished migrating 3 changelog revisions; change in size: 0 bytes
> +  finished migrating 9 total revisions; total change in store size: 0
> bytes
> +  copying phaseroots
> +  data fully migrated to temporary repository
> +  marking source repository as being upgraded; clients will be unable to
> read from repository
> +  starting in-place swap of repository data
> +  replaced files will be backed up at
> $TESTTMP/upgradegd/.hg/upgradebackup.* (glob)
> +  replacing store...
> +  store replacement complete; repository was inconsistent for *s (glob)
> +  finalizing requirements file and making repository readable again
> +  removing old repository content$TESTTMP/upgradegd/.hg/upgradebackup.*
> (glob)
> +  removing temporary repository $TESTTMP/upgradegd/.hg/upgrade.* (glob)
> +  $ hg verify
> +  checking changesets
> +  checking manifests
> +  crosschecking files in changesets and manifests
> +  checking files
> +  checked 3 changesets with 3 changes to 3 files
> +
>    $ cd ..
>
> -
>  store files with special filenames aren't encoded during copy
>
>    $ hg init store-filenames
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20190805/07cad19b/attachment.html>


More information about the Mercurial-devel mailing list