Things we ought to do to improve our packaging

[Matt Mackall]

On Tue, 2013-08-06 at 14:15 -0500, Steve Borho wrote:
> 
> 
> 
> On Tue, Aug 6, 2013 at 2:01 PM, Augie Fackler <raf at durin42.com> wrote:
>         On Tue, Aug 06, 2013 at 01:53:34PM -0500, Matt Mackall wrote:
>         > [this discussion needs to be cross-posted]
>         >
>         > We've got a few long-standing problems with our current
>         packaging:
>         >
>         > - lag official release
>         > - not automated
>         > - not reproduceable by third parties
>         > - not signed
>         > - lack nightly builds
>         > - notable missing platforms (RHEL)
>         > - not uniform in configuration
>         >
>         > Our attitude to date has been "well, we're not really in
>         charge of that,
>         > that's done unofficially by volunteers". I think we've
>         reached a point
>         > where this needs to start changing.
>         >
>         > In an ideal world, a release would work like this:
>         >
>         > - I tag the release
>         > - I push it
>         > - Automated builders kick off on a bunch of VMs or hosts
>         > - Builders run stock makefile targets or scripts in tarball
>         > - Packages get signed
>         > - Packages get uploaded to one or more locations, including
>         > mercurial.selenic.com
>         > - Download links get updated
>         >
>         > Importantly, this same process would also be used to produce
>         automated
>         > nightly builds for testing and bug reporting and discovering
>         packaging
>         > problems before release dates.
>         >
>         > The job of packagers would thus shift from "(manually?)
>         building and
>         > uploading binaries" to "making sure the committed build
>         rules are
>         > correct and up-to-date".
>         >
>         > I think this also needs to be inclusive of builds for
>         packaging systems
>         > (Debian, Ubuntu, RHEL, Centos, Solaris, Pypi, etc.). These
>         actually
>         > suffer from more or less the same problems even though the
>         distribution
>         > model is different. So we should be building nightlies and
>         release
>         > builds for these systems and be more directly responsible
>         for quality
>         > and timeliness here as well.
>         
>         
>         This honestly sounds like a great improvement, given how stale
>         the
>         packages for Ubuntu tend to be, and how some packagers took it
>         upon
>         themselves in the past to enable all the extensions.
>         
>         
> 
> 
> On the Windows side, everything is automated except for two steps:
>
> 1 - signing packages requires a passphrase to be entered
> 2 - uploading is currently manually done through BB's clunky web
> interface
> 
> 
> If we had an SCP or FTP URL to upload to, and a sane policy for
> deleting old nightlies after a short while, it could be automated.

Well, you already have SCP access, no?

In addition to having a fully automated process that builds releases and
nightlies, I think its also important to get to the build script into
the tree so that people who need to build their own copies for various
reasons (need bugfix, need custom deployment, St. Louis trampled by
kaiju) can do so.

> #1 could be dealt with if the person signing the package didn't mind
> storing their passphrase on disk somewhere.

I think we can worry about that at a later stage, shouldn't treat as
blocking.

-- 
Mathematics is the supreme nostalgia of our time.