Convert extension

This extension is distributed with Mercurial.

Author: several people

1. Overview

The convert extension converts repositories from other SCMs (or even Mercurial itself) into Mercurial repositories, with options for filtering and renaming. It's also useful to filter Mercurial repositories to get subsets of an existing one.

It currently supports the following repository types as sources:

It supports branches and incremental or ongoing conversion, as well as some features for tidying up new repositories, such as pruning parts of the tree and rewriting author names.

RepositoryConversion references other conversion tools or provide conversion recipes.

2. Configuration

Configure your .hgrc to enable the extension by adding following lines:

[extensions]
hgext.convert=

3. Usage

hg convert [OPTION]... SOURCE [DEST [REVMAP]]

SOURCE points to the data which are to be imported. It can be:

DEST is a local directory name where the conversion data will go to (Mercurial repository will be created or updated). If DEST is not provided, it will be created by adding -hg suffix to the source directory name. For example if source is in /my/cvs/dir, default destination is /my/cvs/dir-hg. When URLs are supplied, repository name is inferred from the last path component, http://foo.bar/repo/trunk gives trunk-hg.

REVMAP is a a simple text file that maps each source commit ID to the destination ID for each revision. Unless specified, it defaults to the .hg/shamap in the destination directory. This file is automatically created and updated on each commit copied, its purpose is to track which commits were already imported and which were not - and thanks to it allow to resume interrupted import and to make incremental updates.

/!\ When converting to Mercurial, the destination working directory is used as a temporary storage for file revisions but is not updated. hg status lists all these temporary files as unknown. Purge them and update to get a correct view of the converted repository.

3.1. Options

3.1.1. --rev

Convert can optionally stop importing at a given revision, if the --rev option is provided. The argument should be given in terms the source understands (e.g. a revision number for Subversion sources, or a hash for git sources). Revisions newer than specified by this parameter are not imported.

This can also be useful to do incremental conversions. Incremental conversions may be useful not only when tracking newer changes in the source repo but also in very huge repos which would need huge resources for a whole conversion and can be better handled with an incremental one.

Is it possible to convert the last 100 revisions, because the old stuff is not interesting any more???

3.1.2. --authors

Convert can also remap author names during conversion, if the --authors option is provided. The argument should be a simple text file maps each source commit author to a destination commit author. It is handy for source SCMs that use unix logins to identify authors (eg: CVS). Example:

john=John Smith <John.Smith@someplace.net>
tom=Tom Johnson <Tom.Johnson@bigcity.com>

3.1.3. --filemap

Convert can also filter or rename files during conversion, when you supply it a mapping via the --filemap option.

The filemap is a file that specifies which files are to be included, renamed, or omitted. By default all files are included (empty filemap means include everything).

Each line can contain one of the following directives:

All paths should be specified as relative paths rooted in the converted directory.

The include directive causes a file, or all files under a directory, to be included in the destination repository, and the exclusion of any other element that's not under an inclusion rule.

The exclude directive causes files or directories to be omitted.

The rename directive renames a file or directory. To rename from a subdirectory into the root of the repository, use . as the path to rename to.

What if conflicting directives are used (say 'include src' and 'exclude src/doc'). Does more specific win, or first, or last?

It is also possible to use comments, comment lines start with #.

Entries are parsed like Unix shell commands and should be quoted accordingly. What does it mean??? Is it possible to use backticks or shell variables???

For example, to import all files except doc subdirectory, but include doc/foo bar.txt and include doc/FAQ renaming it to faq, use:

# Documentation is to be converted to separate repository.
exclude "doc"
include "doc/foo bar.txt"
rename "doc/FAQ" "faq"

3.1.4. --datesort

Normally convert will import revisions in an order that produces the fewest jumps between branches in the commit log. If you want to make the revision order in the destination more closely match that of the parent, use the --datesort flag.

4. Converting from CVS

cvs client and [http://www.cobite.com/cvsps/ cvsps] are prerequisites. On Windows, cvsps requires cygwin. Work is underway to make cvsps work without cygwin - see http://repo.or.cz/w/cvsps/4msysgit.git/.

cvsps compression is not enabled by default, but can be activated in cvspsrc configuration file.

The convert extension requires using CVS "working directory" since it uses cvsps (it is not possible to convert using direct CVS repository URL).

Example conversion:

# Normal CVS checkout (existing checkout may also be used)
cvs checkout -d :pserver:user@repository.host:/repo/path somemodule
# Actual conversion
hg convert somemodule
# New mercurial repo is created in somemodule-hg directory

5. Converting from Darcs

The converter does not currently handle patch conflicts very well. For more detail, see the notes in [http://selenic.com/repo/hg/file/tip/tests/test-convert-darcs tests/test-convert-darcs]. If you encounter this problem, try using the deprecated contrib/darcs2hg.py script instead.

6. Converting from Subversion

See WorkingWithSubversion for examples of Mercurial over Subversion workflows.

Subversion's Python bindings are a prerequisite. The bindings (generated with SWIG) are installed separately on Windows, and can be found on http://subversion.tigris.org/ . The converter works best with local repositories (file://...), so you may want to use svnsync to create one if your access method is remote (especially if it is via http). The source may be a URL or a path to a repository or working directory. The converter supports a few Subversion-specific options:

svn.trunk
Relative path to the trunk (default: "trunk")
svn.branches
Relative path to tree of branches (default: "branches")
svn.tags
Relative path to tree of tags (default: "tags")
svn.startrev

Start subversion revision (as of 963000ed8cac, > 0.9.5). Work for single branches conversions only.

Set these under [convert] in a .hgrc, or on the command line as follows:

hg convert --config convert.svn.trunk=wackoname [...]

/!\ A child mercurial process is spawned to fetch subversion revision information. The heuristic used to find the mercurial executable is simple but good enough for basic setups. If you are running the convert extension from a mercurial installation which is not the primary one (not in $PATH), you should set the $HG environment variable to the location of your mercurial executable or script, so the child process will be spawned with the expected mercurial version.

6.1. More about Subversion URL and Paths Handling

Let's see what we can do to convert http://code.sixapart.com/svn/memcached. trunk can be converted directly with:

hg convert http://code.sixapart.com/svn/memcached/trunk

Here, convert retrieves all revisions down to the first one created in trunk or coming from a copy living somewhere else. What's important to note is *not all revisions* existing in trunk may be converted: if trunk is overwritten at revision A by a branch coming from another part of the repository, the conversion will stop at revision A. http://code.sixapart.com/svn/memcached/trunk is the root URL, only revisions touching files in its subtree will be converted.

Things are different if we run:

hg convert http://code.sixapart.com/svn/memcached

The root URL is http://code.sixapart.com/svn/memcached. The convert extension tries to detect canonical trunk/branches/tags layout automatically. Here, trunk will be detected and added to the modules to convert. It will also detect the branches subdirectory and add its children directories to conversion targets. Starting at these heads, module history is unrolled like when converting trunk, only this time branching information is detected. For instance, if the branches/memcached-1.1.x module is branched from trunk at revision 255, both will be related in the converted repository. Note the do not follow history below root URL still applies but is usually irrelevant with well behaving trunk/branches/tags layouts. Branches are named after their module name, like memcached-1.1.x and this may be preserved by the destination backend (Mercurial backend defaulting to convert.hg.branchnames=1, named branches are created after these module names). Finally, the tags directory is detected automatically, branching points computed and used to tag converted revisions.

If your repository layout differs from the canonical trunk/branches/tags, these can be redefined with convert.svn.trunk, convert.svn.branches and convert.svn.tags configuration options. Values are relative paths from the root URL, like archives/2006/memcached-old for http://code.sixapart.com/svn/memcached/archives/2006/memcached-old.

Local repositories must be converted using local URLs. Under Unix, a local memcached repository would be converted with:

$ hg convert file://`pwd`/memcached-repo/memcached

Under Windows (assuming memcached-repo in c:\dev), it would be:

$ hg convert file:///c:/dev/memcached-repo/memcached

The extension works from checkouts:

$ hg convert memcached

6.2. Working around Network and Bindings Issues

Sometimes, conversion of remote repositories is complicated by poor network connectivity, remote server misconfiguration, SVN bindings issues or bugs in the converter when handling remote sources. Often, you would like to have the SVN repository locally to play and replay the conversion with different parameters or track bugs more quickly. One solution is to mirror it locally using the svnsync script provided along SVN distributions. Documentation about svnsync can be found here: http://svn.collab.net/repos/svn/trunk/notes/svnsync.txt.

7. Converting to Mercurial

The Mercurial back end supports the following configuration options:

hg.usebranchnames

Use named branches to record source branches (default: true)

hg.clonebranches

Create branches as clones of their parent branches. All branches will be created as subdirectories of the convert destination. (default: false)

hg.tagsbranch

Put tags on the given branch (default: default)

An example of using these settings in a .hgrc file is:

[convert]
hg.usebranchnames=0
hg.clonebranches=1


CategoryExtension CategoryConversion