[PATCH 1 of 2] commit: ignore subrepos completely if the '.hgsub' is excluded via -X
Jason Harris
jason.f.harris at gmail.com
Fri Jan 27 06:18:23 CST 2012
On Jan 26, 2012, at 10:31 PM, Matt Mackall wrote:
> On Thu, 2012-01-26 at 12:31 +0100, Jason Harris wrote:
>> # HG changeset patch
>> # User Jason Harris <jason at jasonfharris.com>
>> # Date 1327576926 -3600
>> # Branch stable
>> # Node ID 410354c0ee27ac8afd192f138aa9c205d1d17898
>> # Parent c2e6c5ef45555ff98dd12bef335c40a91eccc390
>> commit: ignore subrepos completely if the '.hgsub' is excluded via -X
>>
>> As a way for users to make sure that their sub repository state is not
>> committed, make it so that -X .hgsub actually obeys the exclusion.
>
> Use case, please. Bear in mind that "create a consistent snapshot of the
> whole project state" is a core Mercurial principle, and including
> subrepos in that snapshot is the obvious manifestation of that
> principle.
The use case is that merging / rebasing in the super-repo causes bad problems
sometimes. I have picked through enough cases at our work where I get calls and
people's repositories are "stuck" or corrupted through sub-repositories. Maybe
someone accidentally rolled back in a sub-repository, maybe rebased in a sub
repository, or did something inadvertent. This has happened around 5 times in as
many months. Usually my procedure for them is to get them to carefully back up
first, and then start exporting the patches in the super repository, then go
through and strip all .hgsubstate / .hgsub changes from these patches and then
get back to some state before all this happened by stripping or cloning up to a
certain rev, then reapply the sanitized patches. This has so far fixed things.
Often some of our developers are working on both the sub-repositories and the
super-repositories at the same time. Ie they work in the same overall clone.
(previously it was highly recommended to always work on different projects in
different clones, but these days we are more likely to work on different
projects all within the same checkout.) So these developers might change some
stuff in the sub-repository, and independently change stuff in the super
repository, and we don't want these tied together (just yet.).
For instance recently we had a major upgrade to one of our sub-repositories,
however there were large instabilities in the changes which needed to be worked
through by the rest of the core team. We were building nightly builds where we
were showing the rest of the people in the company the product so they could
play with it and solicit feedback. Obviously we didn't want to use the version
of the sub-repo which had instabilities. Ie we wanted to keep the substate
always at a location of our choosing for the wider release. For us developers,
well of course we were more bleeding edge and testing various combinations of
things. Some of us were still fixing things in the super repo, some in the
subrepo, and some in both.
Of course we developers could alawys be very careful, but occasionally one of us
makes a change and commits the subrepo state to the super repository. This
happens relatively silently (ie the developer doesn't see any problems and work
continues... However that devleoper has messed up the production build now.) So
the developers must carefully remember to always commit just specific files in
the super repository (doing this avoids the substate tie together), or to always
be in separate clones, or avoid any other operations which would bind the super
to the state of the sub, etc. But in every 100 commits or so, one slips in wrong
or does something wrong and bamm.. A call to Jason or someone else.
(If we had a dedicated gatekeeper that only propagated changes to the main build
it would slow things down since often one or other member will be putting in
this or that feature at the demand of this or that person and if everything has
to go through one choke point then it's sub optimal. We do of course have leads
that retrospectively look at what went in and then sometimes change stuff later
accordingly... But the process isn't choke pointed with a single person, unless
it's getting really close to release time in which case we switch to a different
operating procedures Ie clone off the version for release and only patch in
approved things, etc.)
(Maybe we should be using ivy (or a similar tool) here for package managment,
but looking at the tutorials for ivy I have yet to see a simple one where it
deals with Mercurial repositories, and setting things up without java etc. (We
mainly use C++) Bascially we want a lightweight way to have a super repo, which
is dependent on a sub-repo and we want to controll this dependence and that's
why we used subrepos.
Thus this would be a really handy setting for us, and by the sounds of it
others. Just as eg tagging is a manual thing which we can decide when we want to
tag, this operation allows us to decide when we want to bind the sub-repo to the
super-repo, and let's us work on both, and have builds off eg the super-repo
where we can control which version of the libraries (sub-repo's) we are using
and it's not just always the latest version of the library whatever the state of
that library is at the time.
This all comes back to tight coupling. The problem as I see it, is that this tight
coupling of the super-repo to the sub-repo's is not really desired at all by us.
Again if we had it to do over we would organize things along the lines you
suggest of having in
http://mercurial.selenic.com/wiki/Subrepository#Use_a_thin_shell_repository_to_manage_your_subrepositories
in fact you added these exact recommendations in revision 65 of the wiki page.
To quote you: "They can thus be treated as completely ordinary repositories and
a developer can largely ignore the additional complexities of subrepositories.
Work can continue in these repositories even if their siblings become
unavailable. Recursive commits in build/ are only needed to synchronize changes
between siblings and to tag releases."
And this works perfectly, in fact it's what we need to do most of the time. It's
much more a build configuration issue as to when to sync the libraries
(sub-repos) to the main repo (super-repo). With this switch our users can choose
when to tie things together, exactly like doing a commit to a thin super shell
if we had one. I guess I argued this before.
However, instead of a dedicated setting to turn them off, I was pointing out a
place where we don't handle the -X option consistently. Ie -X .hgsub should act
like the .hgsub file wasn't there. If we the users want to mis use this. Well
that's up to us. I think enough users want to do this, and don't like having to
have something like the following in a script:
mv .hgsub .disabledhgsub
hg ...
mv .disabledhgsub .hgsub
Well that's the motivation that for consistency -X .hgsub should really ignore
this file and then we don't have to jump through the hoops we are currently
jumping through. It also makes it easier to handle in MacHg when I have the
checkbox which states "Commit sub repository states"
That's the motivation... :)
Cheers,
Jason
> This change strikes me as way too involved,
Well the change although it has a big "diff" is actually trivial. It's bascially:
commands.py:
+ # if -X .hgsub is specified then ignore subrepos completely
+ excluded = opts.get('exclude')
+ if excluded and ('.hgsub' in excluded):
+ ui.setconfig('ui', 'excludesubrepos', True)
+
localrepo.py
+ if not self.ui.configbool('ui', 'excludesubrepos'):
<FOLLOWING-BLOCK-SHIFTED-TO-THE-RIGHT>
I had to do some line breaking in the block afterwards to get in under the 80
char line limit. That's why the next patch was to take the whole block as it was
and factor it out...
> especially at this date in
> the cycle. It's also definitely not how we normally do pattern matching.
>
> Also, it seems the file to ignore is actually .hgsubstate.
>
>> diff --git a/mercurial/commands.py b/mercurial/commands.py
>> --- a/mercurial/commands.py
>> +++ b/mercurial/commands.py
>> @@ -1190,6 +1190,11 @@
>> # Let --subrepos on the command line overide config setting.
>> ui.setconfig('ui', 'commitsubrepos', True)
>>
>> + # if -X .hgsub is specified then ignore subrepos completely
>> + excluded = opts.get('exclude')
>> + if excluded and ('.hgsub' in excluded):
>> + ui.setconfig('ui', 'excludesubrepos', True)
>> +
>> extra = {}
>> if opts.get('close_branch'):
>> if repo['.'].node() not in repo.branchheads():
>> diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
>> --- a/mercurial/localrepo.py
>> +++ b/mercurial/localrepo.py
>> @@ -1097,35 +1097,37 @@
>> # check subrepos
>> subs = []
>> removedsubs = set()
>> - if '.hgsub' in wctx:
>> - # only manage subrepos and .hgsubstate if .hgsub is present
>> - for p in wctx.parents():
>> - removedsubs.update(s for s in p.substate if match(s))
>> - for s in wctx.substate:
>> - removedsubs.discard(s)
>> - if match(s) and wctx.sub(s).dirty():
>> - subs.append(s)
>> - if (subs or removedsubs):
>> - if (not match('.hgsub') and
>> - '.hgsub' in (wctx.modified() + wctx.added())):
>> - raise util.Abort(
>> - _("can't commit subrepos without .hgsub"))
>> - if '.hgsubstate' not in changes[0]:
>> - changes[0].insert(0, '.hgsubstate')
>> - if '.hgsubstate' in changes[2]:
>> - changes[2].remove('.hgsubstate')
>> - elif '.hgsub' in changes[2]:
>> - # clean up .hgsubstate when .hgsub is removed
>> - if ('.hgsubstate' in wctx and
>> - '.hgsubstate' not in changes[0] + changes[1] + changes[2]):
>> - changes[2].insert(0, '.hgsubstate')
>> + if not self.ui.configbool('ui', 'excludesubrepos'):
>> + if '.hgsub' in wctx:
>> + # only manage subrepos and .hgsubstate if .hgsub is present
>> + for p in wctx.parents():
>> + removedsubs.update(s for s in p.substate if match(s))
>> + for s in wctx.substate:
>> + removedsubs.discard(s)
>> + if match(s) and wctx.sub(s).dirty():
>> + subs.append(s)
>> + if (subs or removedsubs):
>> + if (not match('.hgsub') and
>> + '.hgsub' in (wctx.modified() + wctx.added())):
>> + raise util.Abort(
>> + _("can't commit subrepos without .hgsub"))
>> + if '.hgsubstate' not in changes[0]:
>> + changes[0].insert(0, '.hgsubstate')
>> + if '.hgsubstate' in changes[2]:
>> + changes[2].remove('.hgsubstate')
>> + elif '.hgsub' in changes[2]:
>> + # clean up .hgsubstate when .hgsub is removed
>> + if ('.hgsubstate' in wctx and '.hgsubstate' not in
>> + changes[0] + changes[1] + changes[2]):
>> + changes[2].insert(0, '.hgsubstate')
>>
>> - if subs and not self.ui.configbool('ui', 'commitsubrepos', False):
>> - changedsubs = [s for s in subs if wctx.sub(s).dirty(True)]
>> - if changedsubs:
>> - raise util.Abort(_("uncommitted changes in subrepo %s")
>> - % changedsubs[0],
>> - hint=_("use --subrepos for recursive commit"))
>> + if subs and not self.ui.configbool('ui', 'commitsubrepos',
>> + False):
>> + changedsubs = [s for s in subs if wctx.sub(s).dirty(True)]
>> + if changedsubs:
>> + raise util.Abort(_("uncommitted changes in subrepo %s")
>> + % changedsubs[0],
>> + hint=_("use --subrepos for recursive commit"))
>>
>> # make sure all explicit patterns are matched
>> if not force and match.files():
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel at selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel
>
>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
More information about the Mercurial-devel
mailing list