[PATCH 4 of 4 V2] obsolete: allow cycles

[Gregory Szorc]

On Thu, Mar 23, 2017 at 6:29 PM, Jun Wu <quark at fb.com> wrote:

> Excerpts from Augie Fackler's message of 2017-03-23 20:59:03 -0400:
> > > For clocks going backwards, we can detect that, say we get
> > > maxdate = max(m.date for m in markers), then if we are going to write a
> > > marker with date < maxdate, we could warn people, and/or write
> > > (date+rand(0 to 1)) as a fake date. That would at least make
> non-exchange
> > > experience always expected.
> >
> > Could we default to using the current timestamp, but if there’s a member
> > of the obsmarker chain we’re about to add to with a time ahead of our
> > clock, we do (that time + rand(0,1)) as the time? Greg, does that
> > alleviate most of your concern around using times?
>
> That was what I suggested. A warning is probably worthy if that happens.
>
> More strictly, when loading the obsstore, if any marker has a time greater
> than the local system time, it may be worth a warning so we can detect
> problems early.
>

That does mitigate the impact of bad future times somewhat. I think there
are still issues with bad past times losing out precedence. But I could be
wrong since I don't know the full semantics of obsolete store processing.


>
> > (This has the negative that in the presence of marker cycles time is
> > somewhat a lie, but I think the overall ergonomics seem to be about right
> > at least as a thought experiment...)
>

I have a thought experiment to add.

The obsolete store is strictly ordered. So for a repo detached from the
world and not doing any exchange, "date" is strictly metadata and the
offset of a marker in the store is all that matters for precedence.

This assumption goes completely out the window once exchange is involved
because when you do `hg pull` you find all markers from the remote not in
the local store and append them. So now your local store is unordered and
we have this problem of needing to reorder markers to establish precedence.

It seems like we're shooting ourselves in the foot by consolidating all
markers into a single local store. Have we ever considered having a
separate store per remote or at least preserving the illusion of a separate
store (e.g. by recording metadata such as "markers N to M pulled from X")?
This would preserve properties of strict ordering within each peer. There
are still problems of figuring out the relative ordering between markers
produced on different peers (with different clocks). But without online
coordination at the time of marker generation, this is *impossible* to
achieve. The best you can do is preserve strict ordering within each source
and exchange/record generation numbers (e.g. marker number/offset) to
attempt to resolve relative ordering and to drive "merging."

Of course, the implementation of this is complex and requires the concept
of a persistent and identifiable peer. But it is more robust than relying
on clocks alone.

Without an online distributed system coordinating strict ordering, it is
impossible to automatically resolve marker ordering with 100% accuracy. I
worry that attempting to use wall clock times to resolve ordering will
result in silent or non-obvious data loss. That breaks a fundamental
property of version control: don't lose (or corrupt) my data. Whatever we
do here, there needs to be an option to manually resolve ambiguous ordering
without relying on dates. I concede this is a very difficult problem on
several fronts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20170326/24d9424f/attachment.html>