clone fails if target is locked [and a bit on I/O ordering rules]

Bryan O'Sullivan bos at
Tue Aug 16 10:11:23 CDT 2005

We discussed this on IRC last night.  The clone-by-hardlink code has
been somewhat wrong all along, but not in ways that you'll see if you're
not torture testing (which I was).

First off, it attempts to hardlink everything under .hg.  This will
clearly fail in the case of .hg/lock, which is a symlink, hence the
first exception I posted last night.

Secondly, it does the hardlinking in no particular order.  It *might* be
alphabetic, or it might not.  However, in order to satisfy Mercurial's
I/O ordering rules, it *must* link some files in a particular order.

Because Mercurial's metadata files are append-only, it should not be
necessary to lock when doing a clone at all.

When Mercurial is writing metadata, it does the updates in this order:

- Files affected by the operation
- The manifest file
- The changeset file

Readers are supposed to operate in this order:

- The changeset file
- The manifest file
- Data files

If a writer wants to append to a metadata file with a link count above
1, it rewrites the entire file and unlinks the original.  This maintains
separation between hardlinked repositories.

Since the changeset file points to valid parts of the manifest file, and
the manifest file points to valid parts of data files, and the whole
thing is append-only, it's easy to see that breaking hardlinks and
enforcing this ordering gives consistent results, at least between
processes on a single system.

But the hardlinking code is doing the hardlinks willy nilly, and
sometimes the lack of ordering means that the changeset or manifest
point to bits of file that don't exist.  This can only happen if the
repo being cloned is being written during the clone.

The fix is not to read-lock the cloned repository, but to have the
hardlink code honour the ordering rules.


More information about the Mercurial mailing list