Bug 5675 - ShareExtension does not avoid copying subrepositories
Summary: ShareExtension does not avoid copying subrepositories
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: 2.8.2
Hardware: PC Linux
: wish feature
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-06 10:54 UTC by kpop
Modified: 2017-10-31 00:00 UTC (History)
3 users (show)

See Also:
Python Version: ---


Attachments
Minimal example (749 bytes, text/plain)
2017-09-07 04:04 UTC, kpop
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kpop 2017-09-06 10:54 UTC
The ShareExtension is great for reducing disk usage when several checkouts of the same repository are made:

>> hg share Repo WC1
>> hg share Repo WC2

The resulting WC1 and WC2 will typically be a fraction of the size of the Repo, because the .hg/store is only kept once, in Repo.

However, for subrepositories, this is not the case and the store folder is copied. So there is no WC1/.hg/store, but there is a WC1/SubRepo/.hg/store.

Request: can the sharing also be applied to subrepositories?
Comment 1 kpop 2017-09-07 04:04 UTC
Created attachment 1972 [details]
Minimal example

Added minimal example that shows that the main repository actually shares the storage, but not the subrepositories.
Comment 2 Yuya Nishihara 2017-09-07 09:51 UTC
Maybe share can have -S/--subrepo option, which will make the subrepos
(at tip) shared as well. This and -U/--noupdate will be mutually exclusive.
Comment 3 Matt Harbison 2017-09-08 09:45 UTC
(In reply to Yuya Nishihara from comment #2)

I'd like this feature too, but I think we should consider the current behavior broken, and skip the -S.  Consider:

  $ hg clone https://... myclone
  $ hg share myclone myshare
  $ echo 'edits' > myshare/subrepo/file.txt
  $ hg -R myshare commit -S -m 'edits'

At this point, 'myclone' is broken, because the commit to 'myshare' has a .hgsubstate reference that 'myclone' sees, but that aren't in its subrepos.  If you push from 'myshare', it all goes out to the https://... location.  The only fix is to push each subrepo individually.  I can't imagine anyone relies on this behavior.  We go to great lengths to ensure non-broken repos (see issue2314), and I know share + subrepos didn't work at all at some point, so I think this was an oversight.

OP: You might want to look at "clone pooling" as an interim workaround.  It's on my list of things to look at, so I'm not positive it will work, but see `hg help -e share`.
Comment 4 HG Bot 2017-10-19 02:35 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/68e0bcb90357
Matt Harbison <matt_harbison@yahoo.com>
subrepo: share instead of clone if the parent repo is shared (issue5675) (BC)

Previously, only the top level repo was shared, and then any subrepos were
cloned on demand.  This is problematic because commits to the parent repo would
write an updated .hgsubstate to the share source, but the corresponding subrepo
commit would be stuck in the local subrepo.  That would prevent an update in the
source repo.  We already go to great lengths to avoid having inconsistent repos
(e.g., `hg push -r rev` will push _everything_ in a subrepo, even if it isn't
referenced in one of the parent's outgoing commits).  Therefore, this seems like
a bug fix, and there's no option to get the old behavior.  I can't imagine the
previous behavior was useful to anybody.

There shouldn't be an issue with svn, since it is centralized.  Maybe --git-dir
can be used for git subrepos, but I'll leave that to someone more familiar with
git.

An integer was previously being implicitly returned from commands.share(), which
caused dispatch() to start crashing when changing over to returning the shared
repo.  All error paths appear to raise, so this can be hardcoded to success.
The clone command checks for 'is None' in a similar pattern, but since
hg.clone() always returns a tuple, that seems wrong?

.. fix:: Issue 5675

   Creating a share of a repository with a Mercurial subrepository will now
   share the subrepository.

and

.. bc::

   Mercurial subrepositories are now shared instead of cloned when the parent
   repository is shared.  This prevents dangling subrepository references in the
   share source.  Previously shared repositories with cloned subrepositories
   will continue to function unchanged.

(please test the fix)
Comment 5 kpop 2017-10-23 03:52 UTC
With the minimal example, hg creates the file "Subrepo/.hg/sharedpath" instead of "Subrepo/.hg/store" directory. 

For a production repository, sub-repositories are correctly shared with their local source repositories, resulting in a lower disk usage. 

Note that sharing remote repositories makes no sense and results in a clear errormessage as expected.
Comment 6 Bugzilla 2017-10-31 00:00 UTC
Bug was set to TESTING for 7 days, resolving