[PATCH 3 of 4] lfs: add an experimental knob to disable blob serving

Matt Harbison mharbison72 at gmail.com
Sat Mar 31 19:33:49 EDT 2018


On second thought, I can probably just not wrap the wire protocol function.  But the other patches here shouldn’t depend on this.

> On Mar 31, 2018, at 6:28 PM, Matt Harbison <mharbison72 at gmail.com> wrote:
> 
> # HG changeset patch
> # User Matt Harbison <matt_harbison at yahoo.com>
> # Date 1522524043 14400
> #      Sat Mar 31 15:20:43 2018 -0400
> # Node ID f3e750cdf0b3fb1977b7ea0f7685d2a86d3c199c
> # Parent  40f0b221aee73bca2072812062d0f112a134bedf
> lfs: add an experimental knob to disable blob serving
> 
> The use case here is the server admin may want to store the blobs elsewhere.  As
> it stands now, the `lfs.url` config on the client side is all that enforces this
> (the web.allow-* permissions aren't able to block LFS blobs without also
> blocking normal hg traffic).  The real solution to this is to implement the
> 'verify' action on the client and server, but that's not a near term goal.
> Whether this is useful in its own right, and should be promoted out of
> experimental at some point is TBD.
> 
> Since the other two tests that deal with LFS and `hg serve` are already complex
> and have #testcases, this seems like a good time to start a new test dedicated
> to access checks against the server.  I believe that accessing some random URI
> (that isn't a protocol API) will 404 before the access checks, so that's what I
> did here, for simplicity.
> 
> diff --git a/hgext/lfs/__init__.py b/hgext/lfs/__init__.py
> --- a/hgext/lfs/__init__.py
> +++ b/hgext/lfs/__init__.py
> @@ -166,6 +166,9 @@ testedwith = 'ships-with-hg-core'
> configtable = {}
> configitem = registrar.configitem(configtable)
> 
> +configitem('experimental', 'lfs.serve',
> +    default=True,
> +)
> configitem('experimental', 'lfs.user-agent',
>     default=None,
> )
> diff --git a/hgext/lfs/wireprotolfsserver.py b/hgext/lfs/wireprotolfsserver.py
> --- a/hgext/lfs/wireprotolfsserver.py
> +++ b/hgext/lfs/wireprotolfsserver.py
> @@ -35,12 +35,22 @@ def handlewsgirequest(orig, rctx, req, r
>         return False
> 
>     try:
> +        serve = rctx.repo.ui.configbool('experimental', 'lfs.serve')
> +
>         if req.dispatchpath == b'.git/info/lfs/objects/batch':
> +            if not serve:
> +                res.status = hgwebcommon.statusmessage(404)
> +                res.setbodybytes(b'')
> +                return True
>             checkperm(rctx, req, 'pull')
>             return _processbatchrequest(rctx.repo, req, res)
>         # TODO: reserve and use a path in the proposed http wireprotocol /api/
>         #       namespace?
>         elif req.dispatchpath.startswith(b'.hg/lfs/objects'):
> +            if not serve:
> +                res.status = hgwebcommon.statusmessage(404)
> +                res.setbodybytes(b'')
> +                return True
>             return _processbasictransfer(rctx.repo, req, res,
>                                          lambda perm:
>                                                 checkperm(rctx, req, perm))
> diff --git a/tests/test-lfs-serve-access.t b/tests/test-lfs-serve-access.t
> new file mode 100644
> --- /dev/null
> +++ b/tests/test-lfs-serve-access.t
> @@ -0,0 +1,67 @@
> +#require serve
> +
> +  $ cat >> $HGRCPATH <<EOF
> +  > [extensions]
> +  > lfs=
> +  > [lfs]
> +  > url=http://localhost:$HGPORT/.git/info/lfs
> +  > track=all()
> +  > [web]
> +  > push_ssl = False
> +  > allow-push = *
> +  > EOF
> +
> +Serving LFS files can experimentally be turned off.  The long term solution is
> +to support the 'verify' action in both client and server, so that the server can
> +tell the client to store files elsewhere.
> +
> +  $ hg init server
> +  $ hg --config "lfs.usercache=$TESTTMP/servercache" \
> +  >    --config experimental.lfs.serve=False -R server serve -d \
> +  >    -p $HGPORT --pid-file=hg.pid -A $TESTTMP/access.log -E $TESTTMP/errors.log
> +  $ cat hg.pid >> $DAEMON_PIDS
> +
> +Uploads fail...
> +
> +  $ hg init client
> +  $ echo 'this-is-an-lfs-file' > client/lfs.bin
> +  $ hg -R client ci -Am 'initial commit'
> +  adding lfs.bin
> +  $ hg -R client push http://localhost:$HGPORT
> +  pushing to http://localhost:$HGPORT/
> +  searching for changes
> +  abort: LFS HTTP error: HTTP Error 404: Not Found (action=upload)!
> +  [255]
> +
> +... so do a local push to make the data available.  Remove the blob from the
> +default cache, so it attempts to download.
> +  $ hg --config "lfs.usercache=$TESTTMP/servercache" \
> +  >    --config "lfs.url=null://" \
> +  >    -R client push -q server
> +  $ rm -rf `hg config lfs.usercache`
> +
> +Downloads fail...
> +
> +  $ hg clone http://localhost:$HGPORT httpclone
> +  requesting all changes
> +  adding changesets
> +  adding manifests
> +  adding file changes
> +  added 1 changesets with 1 changes to 1 files
> +  new changesets 525251863cad
> +  updating to branch default
> +  abort: LFS HTTP error: HTTP Error 404: Not Found (action=download)!
> +  [255]
> +
> +  $ $PYTHON $RUNTESTDIR/killdaemons.py $DAEMON_PIDS
> +
> +  $ cat $TESTTMP/access.log $TESTTMP/errors.log
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D525251863cad618e55d483555f3d00a2ca99597e x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=phases x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=listkeys HTTP/1.1" 200 - x-hgarg-1:namespace=bookmarks x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
> +  $LOCALIP - - [$LOGDATE$] "POST /.git/info/lfs/objects/batch HTTP/1.1" 404 - (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=capabilities HTTP/1.1" 200 - (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=batch HTTP/1.1" 200 - x-hgarg-1:cmds=heads+%3Bknown+nodes%3D x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
> +  $LOCALIP - - [$LOGDATE$] "GET /?cmd=getbundle HTTP/1.1" 200 - x-hgarg-1:bookmarks=1&bundlecaps=HG20%2Cbundle2%3DHG20%250Abookmarks%250Achangegroup%253D01%252C02%252C03%250Adigests%253Dmd5%252Csha1%252Csha512%250Aerror%253Dabort%252Cunsupportedcontent%252Cpushraced%252Cpushkey%250Ahgtagsfnodes%250Alistkeys%250Aphases%253Dheads%250Apushkey%250Aremote-changegroup%253Dhttp%252Chttps%250Arev-branch-cache%250Astream%253Dv2&cg=1&common=0000000000000000000000000000000000000000&heads=525251863cad618e55d483555f3d00a2ca99597e&listkeys=bookmarks&phases=1 x-hgproto-1:0.1 0.2 comp=$USUAL_COMPRESSIONS$ (glob)
> +  $LOCALIP - - [$LOGDATE$] "POST /.git/info/lfs/objects/batch HTTP/1.1" 404 - (glob)


More information about the Mercurial-devel mailing list