Bug 1852 - subrepos with relative paths always push/pull relative to default
Summary: subrepos with relative paths always push/pull relative to default
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: unspecified
Hardware: All All
: normal bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-10-03 06:53 UTC by Steve Losh
Modified: 2019-02-15 00:00 UTC (History)
10 users (show)

See Also:
Python Version: ---


Attachments
(34 bytes, text/x-patch)
2010-10-14 09:42 UTC, eikeon
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steve Losh 2009-10-03 06:53 UTC
If you use a relative path for a subrepo in .hgsub:

    subrepo = ../subrepo

And if you have two paths set up in mainrepo/.hg/hgrc:

    [paths]
    default = ssh://server1/repos/mainrepo
    mirror = ssh://mirror/repos/mainrepo

Then running "hg push mirror" in the main repo will attempt to:

  * Push subrepo to ssh://server1/repos/mainrepo/../subrepo/
  * Push mainrepo to ssh://mirror/repos/mainrepo/

If there were changesets to the subrepo that are not present on mirror yet, 
and if the 
mainrepo's .hgsubstate was updated to reference one of them, then 
mirror/mainrepo is now is a 
broken state because the subrepo changes were never pushed to 
mirror/mainrepo.

This happens when manually specifying a path with the command as well:

$ hg paths
default = ssh://hg@bitbucket.org/sjl/hgtip/

$ hg push -v ssh://webf/repos/hgtip
running ssh webf "hg -R repos/hgtip serve --stdio"
pushing to ssh://webf/repos/hgtip
pushing subrepo content/tips
running ssh hg@bitbucket.org "hg -R sjl/hgtip/../hgtip-content serve --
stdio"
...
Comment 1 Matt Mackall 2009-10-03 09:20 UTC
For the record, this is the currently documented behavior. See the 5th and
6th bullets at http://mercurial.selenic.com/wiki/subrepos.

Pull is problematic here because it is delayed until update, push follows
its example.
Comment 2 Steve Losh 2009-10-03 09:38 UTC
The fifth bullet point:

"Updating always uses the URL from .hgsub (or any default you've specified in
the subrepo's hgrc), rather than any you might have specified in your last
pull. Pull -r will not filter revisions pulled in subrepositories."

I have:

project/
  .hg/
    hgrc
  .hgsub
  asubrepo/
    .hg/
      hgrc


Contents of project/.hg/hgrc:

[paths]
default = ssh://myserver/repo
mirror = ssh://mirror/repo


Contents of .hgsub:

asubrepo = ../asubrepo


Contents of project/asubrepo/.hg/hgrc: nothing (or no default paths).


Running 'hg push -v mirror' (in the main repo) shows that the subrepo is being
pushed to ssh://myserver/repo/../asubrepo and *not*
ssh://mirror/repo/../asubrepo while the main repo is being pushed to
ssh://mirror/repo.

The subrepo doesn't have a default path in its hgrc, so it seems like the
first part of the bullet point should apply: "Updating always uses the URL
from .hgsub".

If the URL in .hgsub is a relative one shouldn't the effective URL be [url I'm
pushing the main repo to]/[relative url in .hgsub]? Right now it looks like
the effective URL is [default url for the main repo no matter what I'm pushing
to]/[relative url in .hgsub].
Comment 3 Matt Mackall 2009-10-03 09:49 UTC
You write:

If the URL in .hgsub is a relative one shouldn't the effective URL be [url I'm
pushing the main repo to]/[relative url in .hgsub]? Right now it looks like
the effective URL is [default url for the main repo no matter what I'm pushing
to]/[relative url in .hgsub].

--

First let's consider this from the point of view of pull and subsequent
update. The path from pull is not remembered, so only the default url is
available at update time. Thus, we always base relative pulls on the default
path.

For symmetry, push uses the same scheme.

Obviously, this is suboptimal, but I haven't come up with a fix that isn't a
gross hack.

See related issue1849.
Comment 4 Ethan Tira-Thompson 2010-08-30 21:11 UTC
Hi, I've just run into this as well, it seems quite bad... if I specify a repo 
to push into, I expect the subrepos to push there as well, not still go back 
to wherever it was pulled from :(

Is there a workaround?  If I want to change where I am pushing I have to 
manually edit the .hg/hgrc:paths.default for each subrepo?
Comment 5 Martin Geisler 2010-09-01 15:18 UTC
I'm marking 2024 as a duplicate of this.
Comment 6 eikeon 2010-09-07 19:54 UTC
ditto... what ejtttje said. Great work on hg. looking forward to a fix for 
this issue; else subrepos are not of much use for my use case.
Comment 7 Martin Geisler 2010-09-08 01:46 UTC
You guys who actually use subrepositories in production are welcome to go
ahead and try to tackle this.

There is a function called _abssource in subrepo.py which I think can be
used. It currently determines the pull/push path relative to the default
location. I think it just needs to be abstracted a little so that it can do
the same for other pull/push locations.

Please give it a try while I look into other issues with subrepositories.
Comment 8 eikeon 2010-09-17 12:28 UTC
I finally have a bit of time to see if I can help tackle this issue.
Comment 9 Martin Geisler 2010-09-27 06:57 UTC
eikeon: that would be awesome!
Comment 10 eikeon 2010-10-14 09:42 UTC
Here's my patch for making the paths relative to dest instead of default.
Comment 11 kiilerix 2010-10-14 10:04 UTC
Some quick comments to the patch:

See http://mercurial.selenic.com/wiki/ContributingChanges and avoid _.

Does it pass the test suite? Also with push and pull and svn subrepos?

The patch needs some new tests.

How about pull -u and update?

Removing :// handling seems like an unrelated change (brodie is working on
patches that will handle that).


Note that another patch has been proposed on
http://www.selenic.com/pipermail/mercurial-devel/2010-October/025135.html .
Comments are welcome.
Comment 12 eikeon 2010-10-14 15:32 UTC
Thank you for the pointer to the ContributingChanges page.

Not sure I know what you mean by 'avoid _'.

The test suite did pass, as I did find and run them. I will start my next 
patch by adding a few test cases.

The '://' handling is related to this issue; as is the propagating of non-
default path information. I took a different approach to the propagating 
issue then was taken in the other patch:

http://www.selenic.com/pipermail/mercurial-devel/2010-October/025138.html

Not sure which approach seems like the way-to-go?

Can I monitor this issue to see brodie's '://' related patches or is there 
somewhere else I should be looking? Looks like I can hold off updating my 
patch and comment on these other pending patches. Or do I work on updating 
my patch in parallel with these other efforts?
Comment 13 Martin Geisler 2010-10-18 05:56 UTC
eikeon: Mads meant you should change 'parent_dest' into 'parentdest' when he
said 'avoid _'. The Mercurial coding style uses no underscore between words
in identifiers.
Comment 14 Matt Mackall 2010-10-25 20:46 UTC
Degrading to bug, known workaround (manually push/pull subrepos)

Mads, where are we at on this?
Comment 15 kiilerix 2010-10-26 06:31 UTC
mpm: From my point of view we are at
http://www.selenic.com/pipermail/mercurial-devel/2010-October/025387.html .

I neither wanted to push a patch I no longer liked nor one that didn't
follow your advice and wasn't much prettier. Sorry for not pushing harder ;-)
Comment 16 HG Bot 2010-10-26 17:00 UTC
Fixed by http://hg.intevation.org/mercurial/crew/rev/5dbff89cf107
Mads Kiilerich <mads@kiilerich.com>
subrepo: propagate non-default pull/push path to relative subrepos (issue1852)
Comment 17 SaintGermain 2011-03-07 10:17 UTC
Issue 2024 which has been marked as a duplicate of this issue is not
resolved by the patch.
I've opened issue 2684 to re-open the issue previously tracked by 2024.
Comment 18 Bugzilla 2012-05-12 09:03 UTC

--- Bug imported by bugzilla@serpentine.com 2012-05-12 09:03 EDT  ---

This bug was previously known as _bug_ 1852 at http://mercurial.selenic.com/bts/issue1852
Imported an attachment (id=1270)
Comment 19 HG Bot 2014-11-19 12:16 UTC
Fixed by http://selenic.com/repo/hg/rev/5bd04faaa3ee
Matt Harbison <matt_harbison@yahoo.com>
run-tests: don't warn on unnecessary globs mandated by check-code.py

When test output is processed, if os.altsep is defined (i.e. on Windows),
TTest.globmatch() will cause a warning later on if a line has a glob that isn't
necessary.  Unfortunately, the regex checking in check-code.py doesn't have this
context.  Therefore we ended up with cases where the test would get flagged with
a warning only on Windows because a glob was present, because check-code.py
would warn if it wasn't.  For example, from test-subrepo.t:

    $ hg -R issue1852a push `pwd`/issue1852c
    pushing to $TESTTMP/issue1852c (glob)

The glob isn't necessary here because the slash is shown as it was provided.
However, check-code mandates one to handle the case where the default path has
backslashes in it.

Break the cycle by checking against a subset of the check-code rules before
flagging the test with a warning, and ignore the superfluous glob if it matches
a rule.  This change fixes warnings in test-largefiles-update.t, test-subrepo.t,
test-tag.t, and test-rename-dir-merge.t on Windows.

I really hate that the rules are copy/pasted here (minus the leading two spaces)
because it would be nice to only update the rules once, in a single place.  But
I'm not sure how else to do it.  I'm open to suggestions.  Splitting some of the
rules out of check-code.py seems wrong, but so does moving check-code.py out of
contrib, given that other checking scripts live there.

There are other glob patterns that could be copied over, but this is enough to
make the current tests run on Windows.

(please test the fix)
Comment 20 Matt Mackall 2015-01-22 15:04 UTC
Bulk testing -> fixed
Comment 21 HG Bot 2019-02-07 12:35 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/87a6e3c953e0
Matt Harbison <matt_harbison@yahoo.com>
subrepo: avoid false unsafe path detection on Windows

Subrepo paths are not normalized for the OS, so what was happening in the
subsequent root path check was:

    root                  -> $TESTTMP\issue1852a\sub/repo
    util.expandpath(...)  -> $TESTTMP\issue1852a\sub/repo
    os.path.realpath(...) -> $TESTTMP\issue1852a\sub\repo

(please test the fix)
Comment 22 Bugzilla 2019-02-15 00:00 UTC
Bug was set to TESTING for 7 days, resolving