D5137: streamclone: new server config and some API changes for narrow stream clones

[pulkit (Pulkit Goyal)]

pulkit created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This patch introduces a new server config `server.stream-narrow-clones` which if
  set to True will advertise that the server supports narrow stream clones.
  
  This patch also pass on the includes and excludes from getbundle command to
  streamclone generation code.
  
  There is a test added to show that the includepats and excludepats are correctly
  passed.
  
  Upcoming patches will implement storage layer filtering for streamclones and
  then we can remove the temporary error and plug in the whole logic together to
  make narrow stream clones working.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5137

AFFECTED FILES
  mercurial/bundle2.py
  mercurial/configitems.py
  mercurial/help/config.txt
  mercurial/streamclone.py
  tests/test-narrow-clone-stream.t

CHANGE DETAILS

diff --git a/tests/test-narrow-clone-stream.t b/tests/test-narrow-clone-stream.t
new file mode 100644
--- /dev/null
+++ b/tests/test-narrow-clone-stream.t
@@ -0,0 +1,39 @@
+Tests narrow stream clones
+
+  $ . "$TESTDIR/narrow-library.sh"
+
+Server setup
+
+  $ hg init master
+  $ cd master
+  $ mkdir dir
+  $ mkdir dir/src
+  $ cd dir/src
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done
+
+  $ cd ..
+  $ mkdir tests
+  $ cd tests
+  $ for x in `$TESTDIR/seq.py 20`; do echo $x > "f$x"; hg add "f$x"; hg commit -m "Commit src $x"; done
+  $ cd ../../..
+
+Trying to stream clone when the server does not support it
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream
+  streaming all changes
+  remote: abort: server does not support narrow stream clones
+  abort: pull failed on remote
+  [255]
+
+Enable stream clone on the server
+
+  $ echo "[server]" >> master/.hg/hgrc
+  $ echo "stream-narrow-clones=True" >> master/.hg/hgrc
+
+Cloning a specific file when stream clone is supported
+
+  $ hg clone --narrow ssh://user@dummy/master narrow --noupdate --include "dir/src/f10" --stream
+  streaming all changes
+  remote: abort: support for narrow stream clones is missing
+  abort: pull failed on remote
+  [255]
diff --git a/mercurial/streamclone.py b/mercurial/streamclone.py
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -531,7 +531,7 @@
             finally:
                 fp.close()
 
-def generatev2(repo):
+def generatev2(repo, includes, excludes):
     """Emit content for version 2 of a streaming clone.
 
     the data stream consists the following entries:
@@ -544,6 +544,10 @@
     Returns a 3-tuple of (file count, file size, data iterator).
     """
 
+    # temporarily raise error until we add storage level logic
+    if includes or excludes:
+        raise error.Abort(_("support for narrow stream clones is missing"))
+
     with repo.lock():
 
         entries = []
diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -1848,6 +1848,10 @@
     Older Mercurial clients only support zlib compression and this setting
     has no effect for legacy clients.
 
+``stream-narrow-clones``
+    Whether the server supports streaming narrow clones or not.
+    (default: False)
+
 ``uncompressed``
     Whether to allow clients to clone a repository using the
     uncompressed streaming protocol. This transfers about 40% more
diff --git a/mercurial/configitems.py b/mercurial/configitems.py
--- a/mercurial/configitems.py
+++ b/mercurial/configitems.py
@@ -1011,6 +1011,9 @@
 coreconfigitem('server', 'streamunbundle',
     default=False,
 )
+coreconfigitem('server', 'stream-narrow-clones',
+    default=False,
+)
 coreconfigitem('server', 'uncompressed',
     default=True,
 )
diff --git a/mercurial/bundle2.py b/mercurial/bundle2.py
--- a/mercurial/bundle2.py
+++ b/mercurial/bundle2.py
@@ -1687,7 +1687,17 @@
     # to avoid compression to consumers of the bundle.
     bundler.prefercompressed = False
 
-    filecount, bytecount, it = streamclone.generatev2(repo)
+    # get the inlcudes and excludes
+    includepats = kwargs[r'includepats']
+    excludepats = kwargs[r'excludepats']
+
+    narrowstream = repo.ui.configbool('server', 'stream-narrow-clones')
+
+    if (includepats or excludepats) and not narrowstream:
+        raise error.Abort(_('server does not support narrow stream clones'))
+
+    filecount, bytecount, it = streamclone.generatev2(repo, includepats,
+                                                      excludepats)
     requirements = _formatrequirementsspec(repo.requirements)
     part = bundler.newpart('stream2', data=it)
     part.addparam('bytecount', '%d' % bytecount, mandatory=True)



To: pulkit, durin42, #hg-reviewers
Cc: mercurial-devel