Internal-changeset concept

Ryan McElroy rm at
Wed Apr 5 09:23:51 EDT 2017

On 4/5/17 11:40 AM, Ryan McElroy wrote:
> On 4/4/17 9:07 PM, Martin von Zweigbergk via Mercurial-devel wrote:
>> On Tue, Apr 4, 2017 at 12:06 PM, Jun Wu <quark at> wrote:
>>> Since most people want a separate hidden storage that handles wider 
>>> cases, I
>>> don't think this incomplete (internal-only) solution worths investment.
>> I can't say I've followed everything in this long discussion
>> (including the multiple threads), but my gut feeling is that I agree
>> with Jun. I really like the idea of separating hiddenness from
>> obsolescence and I feel like introducing that separation would be a
>> good first step. I'm sure there will be plenty of corner cases related
>> to just that piece (Pierre-Yves pointed out the case of pruning and
>> then re-pulling, for example), so getting that in (most likely hidden
>> behind an [experimental] option) and seeing how it works out would be
>> very useful. It does seem like most people agree on that separation
>> being the right thing to do. I'd prefer to get that reasonably done
>> before we spend too much time discussing the next step (unless the
>> next step seems to make the first step obsolete, but it hasn't seemed
>> that way so far).
> Sorry for being late here, I was swamped with interviews at the 
> beginning of the week. Finally getting some space to catch up here.
> I've managed to read the various threads and the back-and-forth here.
> My overall thoughts are:
> (1) I also agree that having a internal-core-hg hiding system that is 
> fast, scalable, and independent of all other concepts is highly 
> desirable. I imagine this being built on top of a bitmap-style system 
> because that's how I can see it being fast, but this is an 
> implementation detail.
> (2) I think that new concepts like "internal-only", "extinct", and 
> "archived" will be able to interact with this internal hiddenness 
> concept however they wish -- and we can try new things without getting 
> wrapped up discussions about whether we're abusing one concept for one 
> of it's side effects (eg, abusing obsolescence markers because they 
> happen to hide things today). I think avoiding abusing obsmarkers this 
> way is also highly desirable, and having a distinct hiding mechanism 
> allows us to avoid this abuse.
> As far as I can tell, everyone is okay with an internal-core-hg hiding 
> concept that their systems will choose how to interact with -- be it 
> obsolescence or arching or whatever else we come up with.
> (3) Thanks to Pierre-Yves for the detailed analysis of verious 
> mechanisms for implementing internal-only changesets. This is super 
> valuable, especially the parts about why a changeset marker alone is 
> insufficient, and the analysis of the work involved to implement 
> various ideas. You went way above and beyond what my question was 
> intended to solicit. I was honestly just looking for something like 
> "yeah, I've thought about this and I think phases could work" or "I 
> think phases won't work because of this problem" -- but you wrote up a 
> design doc! Amazing.
> (4) In response to the design proposal, I agree that it feels 
> over-complicated to me for what I would be trying to accomplish with 
> the feature. I agree with Jun that we can use "dynamic unhiding" for 
> finding the changesets we're interested in, so I don't think we need 
> two new phases, and I think that losing the strict ordering of phases 
> would be too much pain to introduce for this feature. So, my proposal 
> would be a single phase just above "secret" called "internal" that, 
> like secret, would never be exchanged, and would be hidden by default 
> (unless dynamically unhidden via being checked out or directly 
> accessed, for example). I'm against forbidding the user to do things 
> with this changeset -- I always opt to let the user do things, in 
> general, that they ask to do. If they want or run diff or update or 
> whatever to it, sure, let them. The nice thing about having a generic 
> hiding mechanism [item (2) above] is that we don't have to worry about 
> the details of how things are hidden. Overall, I think having a 
> "default hidden phase that doesn't get exchanged like secret" would 
> get us everything we want, and we don't need to introduce new flags or 
> hard concepts into phases. It's just another phase.

Re-reading this, I think it might be confusing that I'm suggesting this 
phase be a generic way to hide things. I'm not. To be clear, I'm  
proposing this new phase as a possible user of the generic hiding system 
-- and only as a way to give shelve (and things like shelve -- eg, 
amend) a non-stripping, non-obsmarker path forward.

Still, see below. I think it's worth shelving this until we have a 
generic hiding system.

> In conclusion, I propose that we shelve (heh) this discussion about 
> specific things that might use hiddenness until we something like #2 
> above, after which we can experiment much more easily in extensions 
> with the right ways to hide and unhide things, without abusing 
> concepts like obsolescence. That way, we can build these things in a 
> clean and composable manner instead of twisting things up like we've 
> been doing with inhibit at FB (painfully, I might add).
> I think Durham recently sent out a new thread about something like 
> concept #2 above, so I'll go read that now and hope it's more or less 
> what I'm thinking :-)
> ~Ryan

More information about the Mercurial-devel mailing list