[PATCH 1 of 2 bfiles] Support pushing to HTTP stores

[Benjamin Pollack]

On Jun 3, 2010, at 6:47 PM, alexandru wrote:

> bfiles.py |  73 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
> 1 files changed, 66 insertions(+), 7 deletions(-)
> 
> 
> # HG changeset patch
> # User alexandru <alex at hackd.net>
> # Date 1275598699 25200
> # Node ID 2bfadd624bb7ed9c29350ee9e5a791c3bbea222d
> # Parent  947a5f96edcebdf693929788d1b13852b288b522
> Support pushing to HTTP stores
> 
> diff -r 947a5f96edce -r 2bfadd624bb7 bfiles.py
> --- a/bfiles.py	Wed Jun 02 19:07:19 2010 -0400
> +++ b/bfiles.py	Thu Jun 03 13:58:19 2010 -0700
> @@ -32,6 +32,7 @@
> import urlparse
> import urllib
> import urllib2
> +import httplib
> import copy
> import inspect
> import fnmatch
> @@ -1598,18 +1599,35 @@
> 
> 
> class httpstore(basestore):
> +    """A store accessed via HTTP"""
>     def __init__(self, ui, repo, url):
> -        self.ui = ui
> -        self.repo = repo
> -        self.url = url
> -        (baseurl, authinfo) = url_.getauthinfo(self.url)
> +        super(httpstore, self).__init__(ui, repo, url)
> +        self.rawurl, self.path = urlparse.urlsplit(self.url)[1:3]
> +        baseurl, authinfo = url_.getauthinfo(self.url)
>         self.opener = url_.opener(self.ui, authinfo)
> 
>     def put(self, source, filename, hash):
> -        raise NotImplementedError('sorry, HTTP PUT not implemented yet')
> +        destdir = os.path.join(self.path, filename)
> +        dest = os.path.join(destdir, hash)
> +        self.sendfile(source, dest)
> +        self.ui.debug('put %s to remote store\n' % source)
> 
> -    def _verifyfile(self, cctx, cset, contents, standin, verified):
> -        raise NotImplementedError('sorry, verify via HTTP not implemented yet')
> +    def sendfile(self, filename, destname):
> +        self.ui.debug('httpstore.sendfile(%s, %s)\n' % (filename, destname))
> +        try:
> +            fd = open(filename)
> +            httpo = httplib.HTTPConnection(self.rawurl)
> +            httpo.request('PUT', destname, fd)
> +            resp = httpo.getresponse()
> +            if resp.status != httplib.CREATED:
> +                raise util.Abort(_('unable to PUT: %s\n') % resp.reason)
> +            else:
> +                self.ui.note(_('[OK] %s/%s\n') % (self.rawurl, resp.getheader('Location')))
> +        except Exception, e:
> +            raise util.Abort(_('%s') % e)
> +        finally:
> +            httpo.close()
> +            fd.close()

Greg: the more I play with this, and with bfiles on Windows, the more I'm thinking that at least the push destinations should be encoded using the fncache naming strategy. (There's an argument for .hgbfiles being that way, too, due to file name length limits on Windows, but I'm happy to discuss that issue separately.) What are your feelings on changing this for HTTP store? What about for the SSH store?

The fncache code also currently only escapes files that are located in .hg/store, which has been frustrating for me on other occasions when I've wanted to reuse the logic for other locations. (E.g., Kiln caches annotation output, which required some copy-paste coding unless we wanted to store the annotation data in .hg/store). What are the feelings on abstracting that code so that it can provide names for files in other directories?

--Benjamin