[PATCH] subrepo: do not push "clean" subrepos when the parent repo is pushed

[Angel Ezquerra]

On Fri, Feb 15, 2013 at 4:49 AM, Matt Harbison <matt_harbison at yahoo.com> wrote:
> On Thu, 14 Feb 2013 01:13:44 +0100, Angel Ezquerra wrote:
>
>> On Thu, Feb 14, 2013 at 1:06 AM, Angel Ezquerra
>> <angel.ezquerra at gmail.com> wrote:
>
> ...
>
>> This is step one in the plan that Matt, Martin and I discussed to
>> improve subrepos during the London sprint.
>
> Is there a brief overview of the plan somewhere?

I wrote a summary on the titanpad that we used during the sprint:

http://titanpad.com/mercurial26

I've taken that and updated it a little with what has been discussed
in this thread and my own thinking since them:

* Discuss ways to improve some of the subrepo pain points - angel, mg
  - add way to pull subrepos with hg pull
  - just push changed subrepos
  - subrepos are eternal
  - We had a discussion with mpm on this and we believe we have a good plan:
      - Add -S/--subrepos flag to hg pull
      - Add cleanstore() method to subrepos which can tell pull/push
if a subrepo store has changes. If not we can ignore it during push
          - The cleanstore method would check a timestamp or sha1 of
the bookmarks, phaseroots and the changelog, but not the
dirstatedirstate
dirstate
        - Updated on clone and push; but also on pull if the store is
already clean
      - Only push not cleanstore() subrepos
      - subrepo.deletable(): if self.clean() and dirstate working dir is clean()
      - fix Merge bug which makes subrepos stay on the working
directory on update even if there are files on the parent that should
go on the folder occupied by the subrepo
          - using deletable() we could remove subrepos from the working dir...
              - however this could cause data loss on update on a
"central" repository containing relative subrepos. On non central
subrepos it would require recloning the subrepo when updating back to
a revision referring to that subrepo.
       - Introduce subrepo caching (this would fix the problem above
and improve pull time considerably when subrepos are deleted)
            - Cloning would need to be smart enough to look on the
subrepo cache. Otherwise it would not be possible to pull from central
repositories that were are at revision -1 (since all subrepos would be
on the cache, and none on the workind directory).

   - Document subpaths patterns that can make relative subrepos work
with bitbucket and google code

Perhaps it would be worth adding this to the wiki?

>> I also am working on step two of the plan, which was to add a
>> "--subrepos" flag to hg push.
>
> Is this your idea about passing (some?) parameters to subrepos [1]?  If
> so, does 'outgoing' need the same method of filtering the option dict [2]
> for consistency?  (I was a bit surprised that outgoing -S passes along the
> --rev option, which causes it to abort in the subrepo with a (parent)
> hash, or lie or abort if given a rev.)  There's also a couple bugs written
> about --addremove not being passed along, so what to pass or not seems
> like a wider (general?) problem.

The way I've implemented it is very similar to how the current
subrepo.get() works. Basically I want hg pull --subrepos to behave as
if you first did "hg pull" and then you did "hg update -r" for every
new revision that "hg pull" brought into the repository. The idea is
to make sure that you are able to update to any new revision without
needing to have any network access (i.e. that the repository is self
contained after doing hg pull --subrepos, as long as it already was
self contained before).

Martin Gesiler helped me with this patch during the sprint and he has
been helping me since then. I'll send a patch soon.

That being said, out of the options that pull takes there are a lot
that do not make sense when pulling from subrepos, particularly --rev,
--bookmark, --branch and specially --rebase. I think that if you need
to do something special while pulling a subrepo perhaps it would be
best to get into the subrepo and do a regular pull, or perhaps use the
onsub extension (which I wish we integrated into mercurial). One thing
that would help there would be to have a way to refer to the parent
paths when pulling from within a subrepo. This would be particularly
handy when using relative subrepo definitions, as you should.

> FWIW, I've got code that seems to be able to honor the -r flag for
> 'outgoing' and 'push' (issue2314) [3], including the case where the parent
> commits an older version of the subrepo after a newer version.  I wasn't
> happy with how I had to hack the option dict after translating --rev, but
> it sounds like you are working in this area anyway.  So I'll try to submit
> this as a (rough) RFC this weekend, and assuming the concept is OK, we
> will probably need to coordinate how options are handed off to subrepos
> for 'push' and 'outgoing'.

I'll try to have a look at your patches when I get some time.

Angel


> [2] http://www.selenic.com/pipermail/mercurial-devel/2011-
> September/034453.html
>
> [3] http://bz.selenic.com/show_bug.cgi?id=2314
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel