Hidden Commits in 4.3

[Ryan McElroy]

On 4/5/17 5:29 AM, Durham Goode wrote:
> On 4/4/17 8:58 PM, Gregory Szorc wrote:
>> On Tue, Apr 4, 2017 at 6:11 PM, Durham Goode <durham at fb.com
>> <mailto:durham at fb.com>> wrote:
>>
>>         There's been a lot of discussion about how to hide and unhide
>>         commits lately [0][1], and I feel the complexity of our current
>>         approach is hurting our ability to reason about it, making it
>>         impossible to make progress.
>>
>>         I would like to formally propose a new pattern for dealing with
>>         hidden commits, along with the concrete steps to getting it
>>     enabled
>>         in core by default by the August release.
>>
>>         The proposal is quite concise, so check out this 1-page
>>     Google doc
>>         for the details and to comment:
>>
>>         https://goo.gl/7DJ9AI
>>

Thanks for the nice concise doc outlining a specific and achievable goal!

Side-note: +1 on Simon's request to put the doc onto the wiki as a plan.

>>
>>         If people find this approach promising, I can commit to
>>     trying to
>>         get this upstream by the August release. So if you have
>>     questions or
>>         concerns, please let me know so I can address them.
>>
>>
>>         [0] see: "obsolete: track node versions"
>>         [1] see: "repo: add an ability to hide nodes in an
>>     appropriate way"
>>
>>
>> I emphatically support your proposal of introducing a 
>> non-obsolescence mechanism for hiding changesets. It solves a very 
>> real and very painful deficiency in the storage model (strips 
>> required to rewrite history) without forcing complicated workflows 
>> and concepts onto users (evolve). As I said at the sprint, the 
>> horrible performance associated with history rewriting without 
>> obsolescence enabled is one of Mozilla's biggest pain points for 
>> large repos. I can't in good faith recommend evolve to all users 
>> because it is too complicated and can lead to unexpected behavior.

I do too! And I'm glad you're on board. As I've argued in a couple of 
threads recently [1][2] I think that we should concentrate on building 
something generic for hiding without the complexity of evolution while 
evolution continues to iterate on its UX and so-forth. We can use this 
generic hiding mechanism inside of core hg with or without obsolescence 
to great effect.

[1] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096240.html
[2] 
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096248.html

>>
>> I think this proposal for hidden changesets solves the performance 
>> problem while not preventing obsolescence/evolve from making progress 
>> in its problem space. It also doesn't seem to be as cognitively 
>> difficult on users.

Totally agree.

>>
>> There are some areas that need flushed out of course. Obviously storage.
>> But I think the more important problems are the user-facing ones. e.g.
>> what happens if a user updates to a hidden changeset and makes a commit?
>> Do we allow that? I think there needs to be some kind of UX to
>> discourage accidental usage of hidden changesets or at least warnings
>> when things become unhidden because that may be unexpected. That's why
>> things in the doc like `hg checkout HASH` automatically unhiding a
>> changeset make me a little nervous. Same with `hg pull` automatic
>> unhiding. It seems like it is almost too easy to create divergence
>> without any helpful mechanism (obsolescence) to recover from it. 

I think that divergence isn't really a concept until evolution comes 
into play. Remember to compare it to what we have now. Today, 
"divergence" is just different topological branches of development that 
one person decided to rewrite. And sure, it might be a bit problematic 
but it's certainly no worse than what we have today -- now you just get 
any stripped commits back as new commits with no warning. With hidden 
commit revival, we can print a warning: "unhiding 5 hidden commits" when 
doing a pull operation, which is definitely a better place than where we 
are now.

What I like about this incremental approach is that it's clearly a step 
in the right direction without jumping all the way into the (still 
incomplete) evolution concept, but which also doesn't block evolution 
from making progress. Evolution is more work to explain today, but it's 
also more powerful.

>> What
>> I'm still trying to contemplate are the workflows where this would most
>> likely occur. If it is only rare scenarios or things we don't want users
>> doing, I think it can be tolerable.
>
> For the purpose of the proposal, I left this scenario out since it's 
> somewhat of an edge case for V1 and users could recover themselves 
> pretty easily. And I feel the solution could be a layer on top of the 
> proposal.
>
> For actual concrete approaches this problem, I see us doing two things 
> (both of which we're already doing within Facebook):
>
> 1. Provide tiered messaging for various actions on hidden commits. If 
> you perform a safe read operation (hg log -r HIDDEN_HASH), do not 
> print a warning. If you perform a write operation that unhides a 
> commit, but in a recoverable way (hg checkout HIDDEN_HASH, or hg graft 
> HIDDEN_HASH), print a mild warning that you are unhiding a commit. If 
> you attempt to perform a non-recoverable write operation (hg push -r 
> HIDDEN_HASH), block it and require the user to explicitly unhide first.

I can confirm that this approach works really well in practice. Even 
when I'm using the "full evolve" experience, I turn on the 
"directaccess" extension because I otherwise find it infuriating that I 
can't run `hg diff -r <HIDDEN REV>` without also passing `--hidden`.

>
> 2. In conjunction with the auto rebase proposal, when you unhide a 
> commit that has an obsmarker, it will have a message on it saying that 
> the commit is old ("amended as XXX"). So it will be obvious in that 
> output that the commit is not the latest version.

This works spectacularly well -- having this output in the log makes it 
really obvious what happened and why certain things are "old versions" 
(obsolete).

>
>>
>> I'm also curious if you intend to add a new repo requirement for this.
>> If not, what's the story for a legacy client interacting with repos with
>> the new hidden mechanism? IMO that experience is bad enough (legacy
>> client would see a bunch of visible changesets that should be hidden)
>> that I think it warrants locking out old clients.
>
> In what situations would a legacy client interact with a repo created 
> by a new client?  If the user downgraded their local Mercurial 
> install? My initial gut feel is that this would be so rare that it 
> would be ok to just let all the hidden commits become visible.  
> Especially since the only time there would be lots of 
> previously-hidden-and-now-visible commits is if the user used the new 
> client for quite a while before switching back to the old client. I 
> think it might actually be a worse experience to put a requires file 
> in, which would make the old client completely unable to use the repo 
> at all.

Let's remember that most deployments don't have the nice centralized 
control that we do. It's probably fairly common for new and old clients 
to use the same repo, over NFS for example, or via a locally installed 
new version of hg that I'm testing out while system mercurial is older.

That being said, I think Durham is right that allowing access from the 
old client is a better experience than blocking them out. If a few extra 
commits show up, that's not as bad as not being able to use the thing at 
all. I don't feel too strongly about this though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20170405/e3248785/attachment.html>