Proposal for detecting history rewriting on shared repos
gregory.szorc at gmail.com
Wed Feb 12 18:20:28 CST 2014
The share extension and workflow is very fragile. If rewriting occurs on
the original repository, there's a good chance shared clones of that
repo will get corrupted. While there is a giant warning in the output of
`hg help share` to warn you about this, Mercurial currently offers
little to no assistance to detect and recover from this.
At Mozilla, our automation was apparently deleting the original source
repo and re-cloning/pulling it. Clones from the original repo still
seemed to work. But under the right circumstances those clones were
getting corrupted - complaining about missing parents, presumably due to
revlogs not matching exactly.
I propose that Mercurial be a bit more robust detecting known issues
with the share extension.
I propose that we introduce a .hg/storeid (or similar) file whose
content is a randomly-generated value. Let's say a UUID. We call this
file/content the "store ID."
The store ID is created at clone time or when a client opens a local
repo that doesn't have a store ID.
The store ID is *not* transferred between repos at clone time. Instead,
each repo has distinct store IDs. Even local filesystem clones will have
separate store IDs.
The store ID is changed whenever the store experiences revlog rewriting
that isn't a transaction rollback. The store ID is thus a guarantee that
any seen revisions in all committed revlogs in the store still exist. If
the store ID changes, all bets about preexisting revlog content are off.
The intended use case for the store ID is for shared repos to detect
incompatible revlog/store changes (but it may not be limited to that -
anyone have other ideas?). When a repo with .hg/sharedpath is opened,
Mercurial will compare the store ID for both repositories. If there is a
mismatch, the client will abort immediately (presumably with an error
message explaining what happened). This will enable clients to
proactively detect bad practices and stop the world from exploding.
The store ID is backwards compatible. Old clients will simply ignore the
existence of this file.
There is a slight performance penalty to this feature, as opening a
shared repo will need to read the contents of 2 new files. But those
files should be small and serviced from the page cache, so I don't see a
major problem here.
The 1 edge case I can think of is how to deal with opening a shared repo
where both repos don't have a store ID. I reckon we should let that
proceed silently, or perhaps with a one-time notification. The reasoning
is that's what old clients do, it's been good enough so far, so why
If adopted, this proposal could be implemented in 2 major phases. Phase
1 involves creating store IDs and consulting them in the shared repo
open code path. Phase 2 involves adding code to modify the store ID when
revlog rewriting occurs. There are immediate benefits to just phase 1.
There are incremental benefits to making the feature/detection more
robust through phase 2.
gregory.szorc at gmail.com
More information about the Mercurial-devel