[PATCH 1 of 5 V2] treemanifest: create treemanifest class

[Matt Mackall]

On Thu, 2015-03-19 at 15:18 +0000, Martin von Zweigbergk wrote:
> On Thu, Mar 19, 2015 at 6:34 AM Martin von Zweigbergk <martinvonz at google.com>
> wrote:
> 
> > On Thu, Mar 19, 2015 at 6:27 AM Augie Fackler <raf at durin42.com> wrote:
> >
> >> On Wed, Mar 18, 2015 at 10:47:16PM +0000, Martin von Zweigbergk wrote:
> >> > On Wed, Mar 18, 2015 at 3:28 PM Matt Mackall <mpm at selenic.com> wrote:
> >> >
> >> > > On Mon, 2015-03-16 at 16:27 -0700, Martin von Zweigbergk wrote:
> >> > > > # HG changeset patch
> >> > > > # User Martin von Zweigbergk <martinvonz at google.com>
> >> > > > # Date 1426435855 25200
> >> > > > #      Sun Mar 15 09:10:55 2015 -0700
> >> > > > # Node ID 0d969d6efeef08935c93afc416bf406def6b8e59
> >> > > > # Parent  567ae53657544744155897ada91f16f8af61ad8a
> >> > > > treemanifest: create treemanifest class
> >> > > >
> >> > > > There are a number of problems with large and flat manifests.
> >> Copying
> >> > > > from http://mercurial.selenic.com/wiki/ManifestShardingPlan:
> >> > > >
> >> > > >  * manifest too large for RAM
> >> > > >
> >> > > >  * manifest resolution too much CPU (long delta chains)
> >> > > >
> >> > > >  * committing is slow because entire manifest has to be hashed
> >> > > >
> >> > > >  * impossible for narrow clone to leave out part of manifest as all
> >> is
> >> > > >    needed to calculate new hash
> >> > > >
> >> > > >  * diffing two revisions involves traversing entire subdirectories
> >> > > >    even if identical
> >> > > >
> >> > > > This is a first step in a series introducing a manifest revlog per
> >> > > > directory.
> >> > > >
> >> > > > This change adds boolean configuration option
> >> > > > experimental.treemanifest. When the option is enabled, manifests are
> >> > > > parsed into a new tree data structure with one tree node per
> >> > > > directory. At this point, it is just a different data structure in
> >> > > > memory; there is still just a single manifest revlog on disk.
> >> > >
> >> > > I think the right way to do this is:
> >> > >
> >> > > - at repo create time, check the flag
> >> > >    - add treemanifest to requires file
> >> > > - at startup, check requires file
> >> > >    - unconditionally use treemanifest if set
> >> > >
> >> >
> >> > I agree with the second step, but I'm not sure about the first.
> >>
> >> He's 100% right, I should have thought of this before you mailed
> >> it. Basically, if you don't put *something* in the requires, older hg
> >> versions will try and operate on the repository and (at best) claim
> >> it's corrupt.
> >>
> >
> > No worries, I did know about the requires thing. I was instead arguing
> > that the repository is *not* corrupt (to an old client either) after this
> > series, so we can add the requires later.
> >
> 
> I hadn't thought of this before, but to show that it's likely to be safe, I
> just ran the test suite while randomly (50/50) using the two types to make
> sure one version can be written with treemanifest and the next be read with
> manifestdict, and vice versa. I checked that it did in fact choose randomly
> (well, not the same every time anyway).

Yeah, Martin's right: there's no on-disk change here (yet) so it's
premature to touch requires. I'd still like to see the experimental flag
bits in a separate patch.

-- 
Mathematics is the supreme nostalgia of our time.