Things we ought to do to improve our packaging

[Steve Borho]

On Tue, Aug 6, 2013 at 2:47 PM, Matt Mackall <mpm at selenic.com> wrote:

> On Tue, 2013-08-06 at 14:15 -0500, Steve Borho wrote:
> >
> >
> >
> > On Tue, Aug 6, 2013 at 2:01 PM, Augie Fackler <raf at durin42.com> wrote:
> >         On Tue, Aug 06, 2013 at 01:53:34PM -0500, Matt Mackall wrote:
> >         > [this discussion needs to be cross-posted]
> >         >
> >         > We've got a few long-standing problems with our current
> >         packaging:
> >         >
> >         > - lag official release
> >         > - not automated
> >         > - not reproduceable by third parties
> >         > - not signed
> >         > - lack nightly builds
> >         > - notable missing platforms (RHEL)
> >         > - not uniform in configuration
> >         >
> >         > Our attitude to date has been "well, we're not really in
> >         charge of that,
> >         > that's done unofficially by volunteers". I think we've
> >         reached a point
> >         > where this needs to start changing.
> >         >
> >         > In an ideal world, a release would work like this:
> >         >
> >         > - I tag the release
> >         > - I push it
> >         > - Automated builders kick off on a bunch of VMs or hosts
> >         > - Builders run stock makefile targets or scripts in tarball
> >         > - Packages get signed
> >         > - Packages get uploaded to one or more locations, including
> >         > mercurial.selenic.com
> >         > - Download links get updated
> >         >
> >         > Importantly, this same process would also be used to produce
> >         automated
> >         > nightly builds for testing and bug reporting and discovering
> >         packaging
> >         > problems before release dates.
> >         >
> >         > The job of packagers would thus shift from "(manually?)
> >         building and
> >         > uploading binaries" to "making sure the committed build
> >         rules are
> >         > correct and up-to-date".
> >         >
> >         > I think this also needs to be inclusive of builds for
> >         packaging systems
> >         > (Debian, Ubuntu, RHEL, Centos, Solaris, Pypi, etc.). These
> >         actually
> >         > suffer from more or less the same problems even though the
> >         distribution
> >         > model is different. So we should be building nightlies and
> >         release
> >         > builds for these systems and be more directly responsible
> >         for quality
> >         > and timeliness here as well.
> >
> >
> >         This honestly sounds like a great improvement, given how stale
> >         the
> >         packages for Ubuntu tend to be, and how some packagers took it
> >         upon
> >         themselves in the past to enable all the extensions.
> >
> >
> >
> >
> > On the Windows side, everything is automated except for two steps:
> >
> > 1 - signing packages requires a passphrase to be entered
> > 2 - uploading is currently manually done through BB's clunky web
> > interface
> >
> >
> > If we had an SCP or FTP URL to upload to, and a sane policy for
> > deleting old nightlies after a short while, it could be automated.
>
> Well, you already have SCP access, no?
>

Yep, it's just a question of whether you want nightlies in the same
location as releases, or in a different folder that was size-managed.


> In addition to having a fully automated process that builds releases and
> nightlies, I think its also important to get to the build script into
> the tree so that people who need to build their own copies for various
> reasons (need bugfix, need custom deployment, St. Louis trampled by
> kaiju) can do so.


So.. building windows frozen packages is fairly convoluted (especially for
thg but the simple hg MSI is no cakewalk) and has a boatload of prereqs to
be installed.

There is a subrepo like main repository:

https://bitbucket.org/tortoisehg/thg-build

It clones all of the prereq repositories.  The key three being hg, thg, and
winbuild (the rest are extensions bundled with thg)

https://bitbucket.org/tortoisehg/thg-winbuild/overview<https://bitbucket.org/tortoisehg/thg-winbuild/wiki/Home>

See the README for that repo for all the hack-tacular parts that go into
building the installer.


> > #1 could be dealt with if the person signing the package didn't mind
> > storing their passphrase on disk somewhere.
>
> I think we can worry about that at a later stage, shouldn't treat as
> blocking.


There's 3-4 people who build these things themselves, but it's a
non-trivial setup to say the least.

-- 
Steve Borho
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20130806/a2bbe0b0/attachment.html>