[PATCH 4 of 4 V2] obsolete: allow cycles

Wed Mar 22 00:25:03 EDT 2017

On Tue, Mar 21, 2017 at 8:44 PM, Jun Wu <quark at fb.com> wrote:

> Excerpts from Gregory Szorc's message of 2017-03-21 19:32:06 -0700:
> > On Tue, Mar 21, 2017 at 3:15 PM, Jun Wu <quark at fb.com> wrote:
> >
> > > Excerpts from Augie Fackler's message of 2017-03-21 18:03:33 -0400:
> > > > > As long as exchange sends dates, markers are globally ordered, and
> they
> > > > > behave no different then the local case.
> > > > >
> > > > > Say Alice has A -> B created on date1, Bob has B -> A with date2.
> If
> > > date2 >
> > > > > date1, Bob wins. The time of pulling or the pull direction does not
> > > matter.
> > > > > Only the global version (date) matters. If date2 == date1, two
> markers
> > > > > "cancel out" and both A and B become visible.  Previously no matter
> > > what the
> > > > > dates are, A and B are both permanently invisible.
> > > >
> > > > I do worry a little about someone with a bogus clock years in the
> > > > future pumping out malicious nodes that can't be overridden, but I
> > > > suppose in that case your "way out" is that you perturb the hash of
> > > > the revision(s) you want to keep. Seem good enough?
> > >
> > > With a bogus clock, power users could also write arbitrary markers
> with a
> > > newer date to solve the issue.
> > >
> >
> > Hold up.
> >
> > Wall clock times cannot be trusted. This is like a cardinal rule of
> writing
> > software in the real world. Yet this patch appears to not only rely on
> > local wall clock times being ordered but also relies on wall clocks from
> > different machines being ordered. That's a double no-no. Sorting by wall
> > clocks will lead to unexpected behavior which in the eyes of an end-user
> is
> > a bug.
> >
> > Sorting by wall clocks is hardly ever a good idea. I have serious
> > reservations about using wall times to determine obsolescence marker
> > precedence.
>
> The date idea was suggested by marmoute at the sprint. I was thinking about
> the physical offsets of markers in "obsstore" before.
>
> Using "date" for sorting is better than using physical offsets, because
> date
> is stored in markers and is a global thing, while offsets are local, which
> means you need to have some special non-trivial logic to deal with
> exchange.
> This reason alone is enough to convince me to switch from offsets.
>
> I'd note that for some systems like Zookeeper, clocks must be treated
> *very*
> seriously. But for the obsstore, clocks are not that important. Because the
> worst case is some visibility issue of commits. No data loss. No repo
> corruption.  No consistency issue. And could be fixed by power commands.
>

Correct me if I'm wrong, but can't incorrect clocks result in the "wrong"
version of a changeset being visible? For example, I'm working on different
machines and pushing changes to a central server. I create divergence then
correct it later via various obsolescence markers. But, because of clocks,
the "wrong" changeset is unhidden and I unknowingly start basing new work
on it instead of the "correct" changeset. I accidentally land a predecessor
because I assumed obsolescence "just worked" and introduce a subtle bug in
the process.

What are the processes in place to prevent this from happening? How will a
power user even know to use a power command to fix the situation?

>
> Not to mention most systems have sane clocks,

This is dangerous to believe.

Anecdotally, I've experienced the following problems on just the machines I
use day-to-day:

* An old battery on a motherboard caused the system clock to go haywire,
drifing so fast that the default ntp configuration failed to readjust the
clock fast enough, leading to clock skew on the order of several minutes.
Other times, the system would boot up without any persisted time, thinking
it was at some epoch. This resulted in mtimes decades in the past.

* Hyper-V attempts to adjust the system clock inside running Linux VMs by
default. Windows was on California time but the Linux VM was on UTC. The
timezones conflicts and every few seconds Hyper-V and something in systemd
would race to fix the system time. The system time bounced around 7 hours
at a time.

* Virtual machines (in Virtualbox IIRC) resumed after hibernate with the
system time from when they were powered down. It took several seconds or
minutes for NTP or some other process to kick in to adjust the VM's clock
to reality.

And these problems all occurred on machines with internet connectivity.
Could you imagine what would happen if they weren't able to communicate
with an NTP server?

and most markers are date
> unrelated - only those cycles matter - cycles are uncommon.
>

OK. Maybe that makes things better. In what scenarios can we get cycles?

>
> I think dsop summaried this up well:
>
>   Mar 13 10:57:26 <dsop> junw: so basically it boils down to: using date is
>   not perefct, it makes the solution easy and elegant and if clocks on
>   computers are wrong, the user might have a non-optimal user experience,
>   but we never loose data
>

Strictly speaking, yes, we never lose data. But I'd argue that using the
wrong data (changeset) inadvertently would be a massive bug and would
create distrust in version control.

>
> I've been thinking about the cycle problem for a long time and don't think
> there is a better solution practically. The current approach (tens of
> lines)
> is probably the most elegant thing I've ever contributed to the list.
> You're
> encouraged to suggest new ideas. But if the new idea is like some fancy
> format change plus some fancy conflict resolution during exchange, which
> sounds like thousands of lines, I think it's reasonable to say no-no to it.
>

I agree the solution is simple, elegant, and probably works most of the
time. But you've built this castle on an unstable foundation. Convince me
it doesn't matter.

As for alternatives, a correct solution needs to refer to the marker it is
replacing. Since cycles should be rare, can we record the full "replaced
marker" data inline in the new marker? In local storage, that could be an
integer offset to the marker. (Honestly, I don't fully understand the
problem space. I just saw "sort by clocks" and all kinds of alarm bells
went off.)

>
> > >
> > > The whole point of the series is to make it possible to reuse hashes.
> > > That's
> > > required for a nature user-experience of "hg unamend", "hg unstrip",
> > > "hg unabsorb" and maybe a general purposed "hg undo", etc.
> > >
> > > Yes, the undo reusing hash behavior may be solved in some other way,
> like
> > > striping the obsstore, or "inhibit". But I believe the obscycle is the
> most
> > > elegant, bug-free way. And it deals with exchange just well.
> > > _______________________________________________
> > > Mercurial-devel mailing list
> > > Mercurial-devel at mercurial-scm.org
> > > https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20170321/d8641698/attachment.html>