Desired use case for obsmarkers / visibility

[Gregory Szorc]

I have a potential use case for obsmarkers / visibility that I want to run
by people to see if it can be supported.

Changesets are pushed to the Firefox repo via a landing service. This
service essentially imports changesets [submitted by the author] and
rebases them onto the repo head.

Today, when your changesets are landed, you need to perform "garbage
collection" on your local repo to remove the old versions of changesets. We
want landed changesets to disappear after `hg pull` picks up the rebased
versions.

This is a pretty straightforward scenario and is supported by obsmarkers
today. If we enabled the writing of obsolescence markers in the landing
service, things would essentially "just work."

Here's where things get a little more complicated.

When changesets are landed to the Firefox repo today, they are first pushed
to an "integration" repository. Logically, this can be modeled as a single
repo divided into public and draft parts. e.g.

o D (draft) (head)
o C (draft)
o B (public)
o A (public) (root)

When our CI says a changeset is "good," it is promoted to public. e.g.

o D (draft)
o C (public) (formerly draft)
o B (public)
o A (public) (root)

Today, when we encounter a "bad" changeset, we perform a backout. e.g.

o D' (draft) (backout of D)
o D (draft)
o C (public)
o B (public)
o A (public) (root)

Given our push velocity, it is common to have intermediary changesets land
before a changeset is identified as "bad." This means there are changesets
between the initial landings and its backout. e.g.

o D' (draft) (backout of D)
o E (draft)
o D (draft)
o C (public)
o B (public)
o A (public) (root)

The repo with the backouts is eventually published and the final history of
the repo is littered with "bad" changesets and backouts. This causes all
kinds of problems for bisection, annotate, file history, etc.

Instead of performing backouts and leaving the final repo history in a
sub-optimal state, we want to instead "drop" "bad" changesets before they
are published. e.g.

o E' (draft) (rebased from discarded D to C)
|     x D (draft) (discarded)
o C (public)
o B (public)
o A (public) (root)

Since we can identify "bad" changesets relatively quickly, this would
enable us to remove the vast majority of backouts and "bad" changesets from
the final, published repo history.

Again, obsolescence as it exists today facilitates this. We can perform
these drops via `hg histedit` (or similar) and the appropriate "prune"
obsmarkers are written so the canonical repo has the appropriate final
history.

However, the way it works today isn't friendly to end-user workflows.

If we were to deploy this, the following would happen:

1) User creates changeset X and submits for landing.
2) Landing service rebases to X' and writes X->X' marker.
3) X' turns out to be bad and is dropped. X'->null marker is written to
convey the prune.
4) User pulls and sees X->X'->null and hides X because its most recent
successor is pruned.
5) User is left wondering what happened to X. They possibly forget they
need to fix and reland X.

This is bad UX. What we want to happen instead is:

a) User pulls after X' drop and X is still visible.
b) Something else happens and some form of X remains visible/accessible to
user

The server can't expose X' because everyone would see it. We have 1 head
per repo and don't want to be exposing random "bad" changesets to everyone.
This seems to rule out the traditional evolve solution of "touch" a
changeset to revive a prune because I'm not sure how we'd send X' to only
the user that cares about it. There's also no way in obsolescence today to
unhide X once it has been obsoleted.

In the obsmarker world of today, the best solution I can think of is
"delete obsmarkers on the server." If we discarded the X->X' marker (or
didn't write it until X' became public), the end-user's original changeset
X wouldn't be hidden on pull because there is no marker on the server
referencing X. But this approach feels hacky and is extra server-side
complexity, which I'd prefer to avoid.

I /think/ the new visibility work proposed by Jun, Durham, and others might
offer some solutions to this problem. Rather than speculate based on my
limited knowledge of that proposal, I am hoping someone with more knowledge
could weigh in more definitively.

It's worth noting that in our proposed workflow, the "integration"
changesets that are rewritten exist in a separate repository that most
people don't pull from. This means we could potentially break some "rules"
about how obsmarkers work since few would notice. But we do pull the
"integration" repo into the "stable" repo. So presumably obsmarkers would
propagate to the "stable" repo and be pulled by people, where they could
cause problems.

Is there a solution to this use case? FWIW, I think a solution would have
use beyond Mozilla's walls: I'm pretty sure a lot of people would love for
the final history of their repo to be cleaner. It's just that today's VCS
tools (including Git), don't handle distributed history rewriting very
gracefully, which discourages people from having nice things.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20171102/ff49960d/attachment.html>