Mercurial as a database backend

Adam B cruxic at gmail.com
Mon Aug 13 10:46:01 CDT 2012


On Fri, Aug 10, 2012 at 6:08 PM, Greg Ward <greg at gerg.ca> wrote:
>
> On 10 August 2012, Adam B said:
> > Hello all,
> >
> > I have a question about Mercurial's internals.  I'm interested in using
> > Mercurial as a database backend for a peer-to-peer application.  One of
> > the
> > main challenges I see with this is maintaining efficiency with hundreds
> > of
> > thousands of records.  Logically each database "record" should be a
> > versioned file in the repository.  However, storing 100k+ files is a
> > really
> > inefficient use of the disk, especially if the records are small.
> >
> > What I wonder is, how difficult would it be to modify Mercurial to store
> > the files in, for example, a SQLite database?  Not the revlog mind you,
> > that would stay as-is.  I'm just talking about the working copy of each
> > file.  Is the Mercurial code base tightly coupled with the file system
> > or
> > is it's interface with file system nicely factored into a few key
> > functions/classes?
> >
> > I would only need to support the essential mercurial functions like:
> > add,
> > remove, commit, revert, push, pull, update, merge, and log.
>
> Version control is not a database.

Agreed - not in the traditional sense.  But it is precisely the
versioning and distributed control that make it attractive for my
needs.  I'm not the first to have this idea.  The Prophet database
(http://syncwith.us/) uses git as the backend although that project
appears to be dead.

>
> revlog is a pretty clever file format when the data is delta-friendly
> and compressible. (E.g. it would suck for encrypted data, but it's
> great for uncompressed images and plain text.) I really think you want
> to peel back a few layers and look directly at revlog.
>
> (And if you want a fast, low-memory, pure C implementation of revlog,
> I've got just the thing for you: clone http://hg.gerg.ca/xrevlog and
> poke around.)

This looks really interesting.  How much has been stripped away?
What's the best way to learn about revlog?

>
>        Greg
> --
> Greg Ward                                http://www.gerg.ca/
> OUR PLAN HAS FAILED STOP JOHN DENVER IS NOT TRULY DEAD
> STOP HE LIVES ON IN HIS MUSIC STOP PLEASE ADVISE FULL STOP


More information about the Mercurial-devel mailing list