[PATCH 4 of 4 V2] clone: add a server-side option to disable full getbundles (pull-based clones)

Gregory Szorc gregory.szorc at gmail.com
Fri May 12 23:08:44 EDT 2017


On Fri, May 12, 2017 at 7:22 PM, Augie Fackler <raf at durin42.com> wrote:

> On Thu, May 11, 2017 at 10:51:23AM -0700, Siddharth Agarwal wrote:
> > # HG changeset patch
> > # User Siddharth Agarwal <sid0 at fb.com>
> > # Date 1494525005 25200
> > #      Thu May 11 10:50:05 2017 -0700
> > # Node ID fa570690257581e3197f8730b6522e80d58a3f45
> > # Parent  052bd5cfe3769b10c64a4a39d9734a2740d44e16
> > clone: add a server-side option to disable full getbundles (pull-based
> clones)
>
> queued, thanks
>
> very nice - I can think of some repositories that should enable
> clonebundles and this new rejection mechanism to save tons of server
> load. :)
>

FWIW, it is common for >95% of hg.mozilla.org's served bytes to be handled
from clone bundles via S3/CDN. Our daily record saw 42,760,071,088,706
of 43,134,314,834,541
(99.13%) of bytes served from S3. Most of this load is Firefox's CI cloning
the Firefox repositories. We make heavy use of volatile "spot" instances
rather than permanent infrastructure, so there's a lot of cloning go on,
even with aggressive reuse of clones on clients.

If we didn't have clone bundles, we'd need to roll our own caching layer
and/or pay a heavy cost for extra server infrastructure. Ironically, clone
bundles has offloaded so much CPU from servers that we can afford to take
computational hits on the server, such as allowing bundle1 clients to
clone/pull from generaldelta repos and leaving full bundle clones enabled.
But I'm still glad we have config knobs to lock out legacy clients at the
first sign of scaling trouble.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20170512/cfe21ecd/attachment.html>


More information about the Mercurial-devel mailing list