Topic + Mozila [was news from the topic experiment]

[Pierre-Yves David]

On 10/01/2016 09:36 PM, Gregory Szorc wrote:
> On Thu, Sep 22, 2016 at 5:26 PM, Augie Fackler <raf at durin42.com
> <mailto:raf at durin42.com>> wrote:
>
>
>     > On Sep 22, 2016, at 8:18 PM, Pierre-Yves David <pierre-yves.david at ens-lyon.org
>     <mailto:pierre-yves.david at ens-lyon.org>> wrote:
>     >
>     >>> I assume this is along the spirit of your 'hg undo' for evolve (that
>     >>> preserves the hash)?
>     >>
>     >> No. We are thinking about using topic to replace bookmark as the recommended
>     >> workflow at fb. People can get confused if local bookmarks point to public
>     >> changesets.
>     >
>     > I would be happy to discuss that ;-)
>
>     As would I. I’ve fielded several user complaints lately that boil
>     down to wanting an ephemeral, user-selected name for each draft
>     commit (a la what we had with mq patch names). I suspect topics
>     might be able to fill that niche as well.
>
>
> Many long-time Mercurial users at Mozilla still use MQ. Having talked to
> many of them, many of their arguments for using MQ all have a common
> theme: "it's simpler." Specifically:
>
> * `hg qseries` provides a concise view of all their "in progress" work
> * Individual commits have names which can be easily referenced - not
> some random 12 character hexidecimal value
> * The complexities of managing many active heads are hidden from them
> (rebase, graft, log -G, etc)
> * They like that they can edit patch files in .hg/patches if the VCS
> gets in the way
>
> It's worth noting that the Firefox repository has N heads spread out
> over M repositories. The Nightly head is in a different repository from
> the Aurora head from the Beta head, etc. While each repository shares
> the same root changeset and can be pulled into a "unified" repository
> with multiple heads, many developers only clone/pull a single repo/head.
> This means their local repo only has a single head by default. And if
> you use MQ, your local clone continues to only have a single head.

Are these multiple heads identified in any way? Do you use named branch 
for this ? otherwise, how does someone can someone identify them when it 
pulls them all of them into a single repository.

> It is important to remember that there is a significant complexity jump
> from a single head to multiple heads. Once you introduce multiple heads,
> you need to understand:
>
> * The concept of a DAG
> * How to find multiple heads
> * How to figure out which head you are on
> * How to switch your working directory to a different head
> * How to copy/move changesets between heads (graft, rebase, etc)
> * How to combine multiple heads (merging)
> * How to push only what you want to push
>
> Mercurial makes many of these things difficult. For example:
>
> * `hg log` shows changesets from multiple heads without any indication
> they are from multiple heads. Contrast with `hg log -G` or `hg log -f`.
> * Existing feature development methods except for MQ (branches and
> bookmarks) lack a command that concisely lists *all* of "my" commits
> (`hg branches` and `hg bookmarks` only list the tip-most changeset and
> don't show the changeset author or description by default).

I agree we need improvement here.

But I'm not sure how that useful it is to be able to see all commits 
details in a single command. I've been using a mix of `hg topics 
--verbose` and `hg stack` for some time and I'm quite happy about it. 
The first one list the open works with some extra data. The second list 
the content of a specify topic.

That said, none of that is showing the exact DAG relationship of these 
bits and that could be useful.

> * `hg rebase` must be explicitly enabled. Some users think this means
> they shouldn't be using it.

I think the main reason it is still and extension is that the UI can get 
confusing. However last year improvement in the handling of default 
destination probably helped a lot there.

I personally also wonder if we could not integrate the main feature 
directly in the merge commands (something like `hg merge --linear`)

The fact it drops named branch is also a significant issue in my opinion 
(really looks like an implementation details/shortcoming that became a 
"feature").

> * `hg push` pushes all heads by default. Great if you are backing up
> your work to a non-publishing repo. Bad if you are trying to land
> something to the canonical repo.

Unfortunatly backward compatibility will be pretty strong in this area. 
We can probably improve the error message and/or provide some "path 
option" to alter this behavior.

> * `hg update` and `hg pull -u`'s heuristic based model can result in
> surprises

Did you checked that recently? Many improvement in the behavior and 
messages have been done a couple version in the past.

> * No easy way to figure out how the current working directory revision
> relates to the overall repo/DAG

`hg summary` is supposed to provide this (but used to be totally buggy, 
and is probably still not great).

> Many of us have installed extensions, aliases, etc on our own machines
> or have rolled these out to our users to pave over these deficiencies.
> That works... where you have control to do that. It doesn't generally
> work in open source: people are at the whim of what Mercurial supports
> out of the box.

+1 here we have a lot to improve.

Some wiki page relevant to these questions:

* https://www.mercurial-scm.org/wiki/FriendlyHGPlan

* https://www.mercurial-scm.org/wiki/ExtensionsDevel

> Many of us also understand the concepts of distributed version control -
> what the DAG is, how to interact with it, etc. It is really easy to lose
> sight of the fact that most users simply want to get stuff done and they
> don't care about the complexities. They will learn the minimum number of
> commands required to accomplish what they set out to do. A version
> control tool is a barrier to them getting actual work done. They will
> choose the mechanism that is simplest and fastest.
>
> Despite all of Mercurial's current deficiencies in this area, it's worth
> noting that I think it's still better off than Git. If the leap from
> single to multiple heads is hard, try introducing remote refs, tracking
> branches, detached HEAD, the reflog, the staging area, etc. And on top
> of that add a convoluted CLI that makes grasping the concepts difficult.
> Many of the hardcore MQ users at Mozilla detest Git because it is too
> complex. Only with a baptism of fire and likely a lot of hand holding do
> they warm up to Git. And - get this - once they learn the power of the
> DAG, of rebasing and interactive history editing, they fall in love.
> They think Mercurial's MQ model is primitive. Then I show them how to do
> "Git like" multiple head development in Mercurial and they're like "oh,
> this is basically the same except there aren't the complexities of
> understanding refs, the staging area, etc - this is quite nice."
>
> So, topics.
>
> I desperately want topics to be a compelling replacement for die hard MQ
> users. That means that topics needs to maintain the simplicity of MQ as
> much as possible. If people start with a single head repository, topics
> needs to be as simple to use as MQ is, even if multiple topics do
> introduce multiple heads in the DAG. Yet topics should also provide the
> features needed by advanced users - those who fully understand how the
> DAG works and how to manipulate changesets within it.
>
> For the beginning user, Mercurial/topics needs to provide:
>
> * A mechanism to show all active lines of work (list the topics) (and
> possibly how they relate to each other and the underlying repository)

I think this is currently covered by `hg topics` and especially `hg 
topic --verbose`.

> * A mechanism to show the current line of work (and possibly how it
> related to the overall repository)

hg stack does that ☺

> * A mechanism to show/search for unfinished changesets
> * A mechanism to provide a human friendly name to a changeset

One of my goal with topic is to make the need for copying hash around gone.

The current 't3' short name for topic is here for that. It is actually a 
temporary replacement for the "revset indexing" notation ('{}' or 
whatever) that got discussed in London in 2015. So ideally we would use 
".{t3}" to refer to changeset in you current topic and "jungle{t3}" to 
refer to changeset in the "jungle" topic.

I've been happy with the system so far but I'm eager for more feedback 
(there is some obvious improvement to be made too).

> I think the existing topics experiment shows a ton of promise in these
> areas. And as a bonus, the transition from a beginning topics user (1
> published head, 1 topic) to more advanced scenarios is much better than
> with MQ because you don't have to transition to different commands (away
> from the q* MQ commands): you simply add more advanced commands like
> "rebase" and "merge" to your skills set without having to relearn
> anything. I also love how topics "fade away" when changesets become
> published. That's such an improvement over having to manually delete
> things later. Just as long as the user knows when their topics disappear...
>
> There are still some areas for improving topics.
>
> 1) `hg log` still shows *all* changesets in the repository. This is
> confusing for users that don't want to be burdened with the complexity
> of multiple heads. I'd like the behavior of `hg log` to only show
> ancestors - and possibly descendants - of the working directory
> changeset when the working directory changeset is unpublished and has a
> topic. If we show descendants, we should delimit the current changeset
> somehow, possibly by making -G or a mode like it the default in this
> scenario. Realistically, I think `hg log` should behave like `hg log -f`
> and `hg log --children` should supplement that with descendants and a
> marking of the wdir changeset. If there is a branch point in the
> descendants, we should automatically add the graphical view because
> showing changesets from multiple DAG heads without the graphical view is
> really confusing. I can also make the argument that the graphical view
> should always be shown so merge commits can be rendered properly.
>
> 2) There are ways to list all topics and to show the "stack" of
> changesets relevant to what I'm working on. This is great. We're still
> lacking a way to easily list and search "all my in progress work." I
> think something like Jordi's `hg wip` alias, Facebook's `hg smartlog` or
> my proposed `hg underway` formalization of these in the official
> distribution would go a long way to addressing this need. I think this
> command is also an excellent place to introduce beginner users to the
> DAG by rendering its graphical shape as part of rendering multiple
> unpublished heads / topics. When people see a concise view of all in
> progress work with the DAG shape, they can start to understand how all
> the changesets relate to one another and how commands like "rebase" work.

I'm not sure if we need a new command. Adding some extra flag to the 
topic listing could do it.

> 3) `hg topics` isn't showing any changeset metadata, such as the hash
> fragment, author, or first N characters of commit message. I like how
> `hg topics` is concise. But it is almost too concise. Not showing the
> author or commit message means users need to type another command to
> figure out what's actually in the topic. Not showing the changeset could
> mean some power users wanting to perform advanced operations like
> grafting a single changeset could have to perform a 2nd command in order
> to get the information they need.

I think keeping it concise by default is good, but having some flag to 
extend the output would make sense.

However I don't see the usage of two simples command with clear semantic 
as a major drawback.

>
> 4) `hg commit` allows the creation of new, label-less heads (no
> bookmark, topic, etc) or the creation of a divergent head on a topic. We
> should think long and hard about disallowing this by default in some
> scenarios (such as when a repo contains a topic authored by the user).
> We should also consider issuing a warning when creating a commit that
> isn't tracked by any label and providing a simple command to be printed
> in that warning that can easily correct that "mistake." (I vaguely
> recall discussions about "divergent" topics at various sprints, but I
> think it may have revolved around push/pull behavior?) The goal here is
> we want to guide beginner users to a single way of doing feature
> development. We want simple commands like `hg topic` to list all their
> in-progress work. We don't want their commits to fall through a crack
> into a "label-less" or "topic-less" world where they need to learn a new
> "advanced" command to find them, especially if the "advanced" command
> introduces them to more concepts, such as the DAG. We want users to be
> introduced to new concepts when they choose to, not when they make a
> mistake like failing to attach a topic to a commit. Power users won't
> like being forced to use topics. We can have a config flag to disable
> this. We can also provide `hg commit --force`.

This is a very important point, how do we get people to use it. I think 
there is a lot room to improve messaging to nudge people toward topic.

I've also been thinking about a mode that prevent local draft changeset 
without a topic.

> 5) `hg topics` feels like it could evolve to do too much. I like `hg
> stack` being in its own world. I wonder if we could further separate the
> querying pieces of `hg topics` from the manipulation pieces. I think it
> is important for commands that make important changes to be isolated
> from commands that just query.

I'm slowly leaning toward this too. I know that we have been moving 
toward "one command for all" in the past years (eg: "bookmarks") but I'm 
not sure this is giving good result here.

> 6) `hg stack` needs some UX love. We could potentially also consolidate
> various functionality for "show relevant changesets" into the same
> command or suite of commands. Although I'd rather we not have a single
> command to do too much or require users to type too many characters or
> require too many concepts. In the `hg log` world, I've thought about a
> "magic log" command where you could have single character prefixes to
> "log" as a shortcut to enabling certain flags. e.g. `hg flog` == `hg log
> -f`, `hg glog` = `hg log -G`, `hg gflog` = `hg log -G -f`. Perhaps we
> could introduce `hg show <what>` to allow showing pre-defined, concise
> views of topics, bookmarks, stack, unfinished work, etc. This would
> avoid command proliferation while delimiting query commands from
> manipulation commands. `hg help show` would also serve as a gateway to
> discover other "views" and ways of doing things. `hg *show` would be
> reserved for shortcuts like `hg tshow` = `hg show topics`. Just a thought.

That's interesting.

In the 'command with prefix" territory, we have 'pdiff' and 'pstatus' 
that are quite useful too (equivalent of qdiff and qstatus in postmq world)

> OK. That's a long post. Hopefully I've given people things to think
> about. I want to emphasize topics has a lot of promise and there's a lot
> to love in its current implementation. I just hope we can leverage the
> opportunity to solve some longstanding shortcomings in the user
> experience in the process of implementing it. I hope we can design the
> experience such that it is enjoyable for existing MQ users and
> especially to new users of version control. If we can provide an
> experience that is compelling for new users and provides an effective
> transition towards advanced features without alienating power users,
> that will be a compelling reason to choose Mercurial over other version
> control tools. I think topics has that potential more than branches,
> bookmarks, and MQ ever did.

Thanks for the long message, really enlighten the discussion. It also 
makes me feel we not doing all that topic work for nothing.
However it is a bit unfortunate that it is a sub-answer to a long and 
old discussion. Maybe it would have been better to make a whole new 
thread with it.

Cheers.

-- 
Pierre-Yves David