[PATCH RFC] repair: add mechanism to convert/upgrade a repo in place

[Yuya Nishihara]

On Mon, 15 Feb 2016 15:47:45 -0800, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1455580051 28800
> #      Mon Feb 15 15:47:31 2016 -0800
> # Node ID d65feeab5bc118778d9523e7df44fe38804046ce
> # Parent  a3fcea8d55f7c2b3e9d83c00cbe303890a906775
> repair: add mechanism to convert/upgrade a repo in place
> 
> Pierre-Yves wants a debug command to "upgrade" repositories to
> generaldelta. Taking a step backward, upgrading a repository to
> generaldelta is a subset of the general task of adding a new
> requirement to an existing repository.
> 
> This patch begins the implementation of a generic, in-place
> repository "upgrade" mechanism that can be performed on a live
> repo with minimal downtime (as opposed to `hg clone` which
> doesn't take out a lock on the source repo and therefore is
> susceptible from repositories gaining new data while operating
> on them).

[snip]

> + at command('debugupgraderepo')
> +def debugupgraderepo(ui, repo):
> +    """upgrade a repository to use different features
> +
> +    During the upgrade, errors may be encountered when reading from the
> +    repository. This command should therefore not be executed on live
> +    repositories.
> +    """
> +    repair.upgraderepo(repo)

You said "hg clone" is susceptible because it doesn't take a lock, but
"debugupgraderepo" is also unusable for live repositories. What's the benefit
of this command?

Even if "debugupgraderepo" takes a lock, another client may see old "requires"
and will go wrong after a lock released.

> +def _upgradestore(repo, requirements):
> +    try:
> +        # It is easier to create a new repo than to instantiate all the
> +        # components separately.
> +        tmprepo = localrepo.localrepository(repo.baseui,
> +                                            path=repo.join('tmprepo'),
> +                                            create=True)
> +
> +        with tmprepo.transaction('upgrade') as tr:
> +            # Start by cloning revlogs individually.
> +            total = 0
> +            for t in repo.store.walk():
> +                if t[0].endswith('.i'):
> +                    total += 1
> +
> +            i = 0
> +            for unencoded, encoded, size in repo.store.walk():
> +                if unencoded.endswith('.d'):
> +                    continue
> +
> +                i += 1
> +                repo.ui.progress('upgrade', i, total=total)
> +
> +                oldrl = revlog.revlog(repo.svfs, unencoded)
> +                newrl = revlog.revlog(tmprepo.svfs, unencoded)
> +
> +                # generaldelta is never enabled on changelog because it isn't
> +                # useful.
> +                if unencoded == '00changelog.i':
> +                    newrl.version &= ~revlog.REVLOGGENERALDELTA
> +                    newrl._generaldelta = False
> +
> +                oldrl.clone(newrl, tr)

Missed a patch for revlog.clone() ?