Cloning locally a repo with subrepos using ../ in .hgsub

[Mads Kiilerich]

Mike Williams wrote, On 01/28/2011 04:23 PM:
> Hi
>
> I am investigating the use of sub-repos to share different subsets of 
> a collection of libraries amongst many projects.  This approach has 
> been mentioned in other mailings to this list so I don't believe it is 
> an oddball requirement.
>
> So, I have created repos of the libraries at the same level as the 
> projects on the server, and then in each project reference the 
> sub-repos for the libraries in the .hgsub files as:
>
> libname = ../libname
>
> As expected this is all working fine when for creating clones from the 
> server.  The pain has started with creating local clones from the 
> clone created from the server - the relative paths no longer hold on 
> the local machine.  The aim is to follow the typical hg workflow of 
> having a clean clones to exchange changesets with the master repos, 
> and local clones to work on.
>
> Adding the subpaths extension and with a minimal .hg/subpaths file 
> containing(*)
>
> [subpaths]
> ../(.*) = \1
>
> I can then clone locally from the master repo clone and everything 
> looks good until I want to interact with the master repos from the 
> original clone.  The simple solution is to rename/move .hg/subpaths 
> somewhere else for the duration of the interactions and then move it 
> back when working locally.
>
> There are two issues here I would like suggestions about:
>
> 1) Is there a better parent/sub-repo structure to represent this many 
> projects/many libraries structure on the server?  We have 10s of 
> projects and 100s of libraries with no reasonable project to "own" a 
> set of the libraries such that the normally recommended .hgsub entry 
> of libname = libname becomes usable.
>
> 2) There does seem to be an idiom here to convert a server's flat 
> sub-repo structure to a hierarchy locally.  Should this be rolled into 
> mercurial of the subpath extension, possibly just to remove leading 
> ../ from .hgsub entries?  This would be global command option 
> (--internal?) so it doesn't change any existing behaviour.
>
> Yes there is a downside with this in that no one is reproducing the 
> repo structure of the server, which makes rebuilding it after an 
> outage an interesting activity in its own right - perhaps a server 
> repo with all the repos being served as sub-repos of it.  But this is 
> disappearing up another line of thought so I will stop now.

In my opinion subrepos are quite usable and stable, but not fully 
feature complete and best practice haven't been established yet. So you 
are right, the right way to solve  your problem isn't obvious.

Much of what you mention and what could be improved in Mercurial was 
also "discussed" in the tread at 
http://selenic.com/pipermail/mercurial-devel/2011-January/027285.html .

With the risk repeating what I said in that thread:

I think subrepos always should be used with trivial mappings as libname 
= libname. You have just given a good description of why other mappings 
is a bad idea. A consequence of trivial mappings is that repositories 
also must be stored "in place" in the tree structure on shared servers. 
You must thus either accept that the subrepos are stored many times on 
the server, or use symlinks or some kind of url rewrite to share the 
storage.

Note that Mercurial - also with subrepos - make working directory 
content 100% reproducible, so when a bug is fixed in a shared library it 
won't automagically show up in all repos that use it no matter what. You 
need to have a workflow for updating to the revision with the bugfix 
anyway, so the simple solution of storing it multiple times might not be 
that bad.

> (*) A little bit of experience from experimenting with the subpaths 
> extension.  The subpath is (currently) as it appears in .hgsub and not 
> normalised in anyway.  If you have mixture of Windows and unix path 
> separators present then you will need an entry of:
>
> ..(\\|/)(.*) = \2

That sounds like a bug. Can you provide a full but minimal example that 
demonstrates the problem?

/Mads