Things we ought to do to improve our packaging

[Javi Merino]

On Wed, Aug 07, 2013 at 05:35:35PM -0500, Matt Mackall wrote:
> On Wed, 2013-08-07 at 08:58 +0200, Dan Villiom Podlaski Christiansen
> wrote:
> > On 06/08/2013, at 21.01, Augie Fackler <raf at durin42.com> wrote:
> > 
> > > On Tue, Aug 06, 2013 at 01:53:34PM -0500, Matt Mackall wrote:
> > >> [this discussion needs to be cross-posted]
> > >> 
> > >> We've got a few long-standing problems with our current packaging:
> > >> 
> > >> - lag official release
> > >> - not automated
> > >> - not reproduceable by third parties
> > >> - not signed
> > >> - lack nightly builds
> > >> - notable missing platforms (RHEL)
> > >> - not uniform in configuration
> > >> 
> > >> Our attitude to date has been "well, we're not really in charge of that,
> > >> that's done unofficially by volunteers". I think we've reached a point
> > >> where this needs to start changing.
> > >> 
> > >> In an ideal world, a release would work like this:
> > >> 
> > >> - I tag the release
> > >> - I push it
> > >> - Automated builders kick off on a bunch of VMs or hosts
> > >> - Builders run stock makefile targets or scripts in tarball
> > >> - Packages get signed
> > >> - Packages get uploaded to one or more locations, including
> > >> mercurial.selenic.com
> > >> - Download links get updated
> > >> 
> > >> Importantly, this same process would also be used to produce automated
> > >> nightly builds for testing and bug reporting and discovering packaging
> > >> problems before release dates.
> > >> 
> > >> The job of packagers would thus shift from "(manually?) building and
> > >> uploading binaries" to "making sure the committed build rules are
> > >> correct and up-to-date".
> > >> 
> > >> I think this also needs to be inclusive of builds for packaging systems
> > >> (Debian, Ubuntu, RHEL, Centos, Solaris, Pypi, etc.). These actually
> > >> suffer from more or less the same problems even though the distribution
> > >> model is different. So we should be building nightlies and release
> > >> builds for these systems and be more directly responsible for quality
> > >> and timeliness here as well.
> > > 
> > > This honestly sounds like a great improvement, given how stale the
> > > packages for Ubuntu tend to be, and how some packagers took it upon
> > > themselves in the past to enable all the extensions.
> > 
> > They still enable the full mergetools.rc list, though. This would be
> > fine if it only listed helpful GUI tools or reasonably user-friendly
> > commands line tools. However, it also includes vimdiff, with its
> > rather steep learning curve…
> 
> This is indeed regrettable. However, the number of people that actually
> want vimdiff is not the empty set, which means taking this out would be
> a regression for _people already comfortable with our tool_, which is
> much worse than having introduced vimdiff in the first place.
> 
> There's basically nothing we can do here other than documentation.
> 
> FYI, the folks at Facebook have come up with a nice script for GUI-less
> systems that fires up $EDITOR at each set of conflict markers that is
> arguably a better user experience than either vimdiff or emerge and
> avoids the usual problems of manually editing after merge. After we get
> some feedback on that, we'll probably drop it in contrib/.
> 
> All of this is actually pretty low on my concern list about packaging
> which runs something like this:
> 
> - OS X
> - lack of nightlies
> - RHEL/Centos availability
> - Debian/Ubuntu lag, lack of backports

Ok, I'm the one to blame for the Debian lag.  You're not the first
upstream complaining about this and I can certainly understand why.  I
think the best solution is to end up with something like what the
mozilla guys did: an unofficial (according to Debian) repository[0]
that has the latest and greatest versions of mozilla packages.  That
way people that want the latest versions as soon as possible can add
that repository.  The rest of the Debian users will get them from the
official Debian repositories with its usual lag but being (in theory)
more stable.

[0] http://mozilla.debian.net/

I'll upload mercurial 2.7 to Debian sid next week, I'll try to come up
with a script that does the basic packaging so that you can autobuild
Debian packages and provide them via mercurial.selenic.com.  Once
that's working, we can talk about making a proper repository that can
be added to apt and autobuilding Ubuntu packages.  Thoughts?

Cheers,
Javi