D1753: streamclone: preserve remote phases (issue5648)

indygreg (Gregory Szorc) phabricator at mercurial-scm.org
Sun Dec 24 13:07:00 EST 2017


indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  As the tests added by the previous changeset demonstrate, stream clones
  were never taught about the existence of phases. As a result, all
  changesets applied via stream clone were made public, even if they
  were draft (or even secret) on the remote.
  
  This commit integrates phases knowledge into stream clones.
  
  As the inline comment explains, because there are scenarios where
  we can't disambiguate phases support, the resulting phase data may
  not match the server exactly. However, I believe the remaining
  areas of buggy behavior are confined to secret changesets. Secret
  changesets aren't common. And transferring them via stream clones
  either requires a buggy Mercurial server or a specially-configured
  server. I think it will require stream clone support in bundle2
  before we can properly handle secret phases for stream clones.
  
  It /might/ be acceptable to force all local changesets to secret
  and then promote to draft or public from remote phases data. However,
  I was having trouble implementing this. Existing phases code doesn't
  seem geared towards supporting this scenario and I didn't want to
  incur a lot of extra work to refactor the phases code. Perfect is
  the enemy of good: I'm inclined to tackle secret phase as part of
  supporting stream clones in bundle2.
  
  This change introduces a `roots(all())` revset, which will require
  a linear scan of the changelog during clone bundles. On a Firefox
  repo, this may slow down stream clones by ~1s. Again, support for
  stream clones in bundle2 should help here.
  
  .. fix::
  
    Draft changeset phase is now preserved during a stream clone.
    
    Before, all changesets pulled via a stream clone would have a
    public phase, even if they were draft on the remote and the remote
    was non-publishing.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1753

AFFECTED FILES
  mercurial/streamclone.py
  tests/test-clone-uncompressed.t
  tests/test-http-bundle1.t
  tests/test-http-proxy.t
  tests/test-http.t

CHANGE DETAILS

diff --git a/tests/test-http.t b/tests/test-http.t
--- a/tests/test-http.t
+++ b/tests/test-http.t
@@ -282,8 +282,9 @@
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=namespaces x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=capabilities HTTP/1.1" 200 -
+  "GET /?cmd=listkeys HTTP/1.1" 401 - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
+  "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=branchmap HTTP/1.1" 200 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
-  "GET /?cmd=stream_out HTTP/1.1" 401 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=stream_out HTTP/1.1" 200 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D5fed3813f7f5e1824344fdc9cf8f63bb662c292d x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&$USUAL_BUNDLE_CAPS$&cg=0&common=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&heads=5fed3813f7f5e1824344fdc9cf8f63bb662c292d&listkeys=bookmarks&phases=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
diff --git a/tests/test-http-proxy.t b/tests/test-http-proxy.t
--- a/tests/test-http-proxy.t
+++ b/tests/test-http-proxy.t
@@ -107,6 +107,7 @@
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cat proxy.log
   * - - [*] "GET http://localhost:$HGPORT/?cmd=capabilities HTTP/1.1" - - (glob)
+  $LOCALIP - - [$LOGDATE$] "GET http://localhost:$HGPORT/?cmd=listkeys HTTP/1.1" - - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   $LOCALIP - - [$LOGDATE$] "GET http://localhost:$HGPORT/?cmd=branchmap HTTP/1.1" - - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET http://localhost:$HGPORT/?cmd=stream_out HTTP/1.1" - - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
   $LOCALIP - - [$LOGDATE$] "GET http://localhost:$HGPORT/?cmd=batch HTTP/1.1" - - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D83180e7845de420a1bb46896fd5fe05294f8d629 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
diff --git a/tests/test-http-bundle1.t b/tests/test-http-bundle1.t
--- a/tests/test-http-bundle1.t
+++ b/tests/test-http-bundle1.t
@@ -291,8 +291,9 @@
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=namespaces x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=capabilities HTTP/1.1" 200 -
+  "GET /?cmd=listkeys HTTP/1.1" 401 - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
+  "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=branchmap HTTP/1.1" 200 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
-  "GET /?cmd=stream_out HTTP/1.1" 401 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=stream_out HTTP/1.1" 200 - x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
   "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D5fed3813f7f5e1824344fdc9cf8f63bb662c292d x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$
diff --git a/tests/test-clone-uncompressed.t b/tests/test-clone-uncompressed.t
--- a/tests/test-clone-uncompressed.t
+++ b/tests/test-clone-uncompressed.t
@@ -39,6 +39,9 @@
   $ hg --debug --config worker.backgroundclose=true --config worker.backgroundcloseminfilecount=1 clone --stream -U http://localhost:$HGPORT clone-background | grep -v adding
   using http://localhost:$HGPORT/
   sending capabilities command
+  preparing listkeys for "phases"
+  sending listkeys command
+  received listkey for "phases": 58 bytes
   sending branchmap command
   streaming all changes
   sending stream_out command
@@ -94,11 +97,9 @@
   searching for changes
   no changes found
 
-TODO this is buggy
-
   $ hg -R all-draft log -T '{rev} {phase}\n'
-  1 public
-  0 public
+  1 draft
+  0 draft
 
 Mixed phase is preserved
 
@@ -110,10 +111,8 @@
   searching for changes
   no changes found
 
-TODO this is buggy
-
   $ hg -R mixed-draft log -T '{rev} {phase}\n'
-  1 public
+  1 draft
   0 public
 
 Cannot stream clone when there are secret changesets
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -10,6 +10,9 @@
 import struct
 
 from .i18n import _
+from .node import (
+    hex,
+)
 from . import (
     branchmap,
     error,
@@ -118,6 +121,8 @@
     repo = pullop.repo
     remote = pullop.remote
 
+    remotephases = pullop.remote.listkeys('phases')
+
     # Save remote branchmap. We will use it later to speed up branchcache
     # creation.
     rbranchmap = None
@@ -158,6 +163,45 @@
         repo._applyopenerreqs()
         repo._writerequirements()
 
+        # Application of phases data is subtly non-trivial.
+        #
+        # The following have to be considered:
+        #
+        # * The remote could be running an old version of Mercurial that
+        #   isn't phases aware. Remote phases data will be empty. (This should
+        #   not be common since phases were supported since Mercurial 1.9.)
+        # * Some versions of Mercurial (before issue 5589 was fixed) can
+        #   serve secret changesets. Versions after can be configured to serve
+        #   secret changesets.
+        # * The remote could be publishing or non-publishing.
+        # * If local changesets are secret, this impacts discovery and other
+        #   mechanisms that occur during the "pull" that is performed after
+        #   this function returns.
+        #
+        # This combination of scenarios means that we can't 100% reliably
+        # reproduce the remote phases state. For example, if the remote is
+        # configured to serve secret changesets and all changesets are secret,
+        # there won't be any phases data and we won't be able to distinguish
+        # between that repo and a Mercurial that isn't phases aware.
+        #
+        # Our strategy is to look at the remote phases data. If there isn't any,
+        # we assume we are talking to an old server that doesn't support phases.
+        # We keep all changesets as public. Otherwise, we force all changesets
+        # to draft. Then we apply the remote phases data.
+        if remotephases:
+            roots = [ctx.node() for ctx in repo.set('roots(all())')]
+            tr = pullop.gettransaction()
+            phases.retractboundary(repo, tr, phases.draft, roots)
+
+            # Strictly speaking, we may not need to apply remote phases here,
+            # as the incremental pull after this function may take care of it.
+            # But since we already have the data, we might as well use it.
+            publicheads, draftroots = phases.analyzeremotephases(
+                repo, repo.heads(), {hex(n): phases.draft for n in roots})
+
+            if publicheads:
+                phases.advanceboundary(repo, tr, phases.public, publicheads)
+
         if rbranchmap:
             branchmap.replacecache(repo, rbranchmap)
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel


More information about the Mercurial-devel mailing list