Strategies for push/merge problem?

[Sean Russell]

On Monday 14 July 2008 09:26:39 pm Sean Kelley wrote:
> > Do you really have 100s of developers in a flat structure with no
> > hierarchy at all?
>
> Yes, we do.  They are all team focused with many product teams.  In
> fact, many of these projects they work on are applications, daemons,
> utilities, vendorsrc and other projects that make up embedded Linux
> products.  We use the a modified version of the PokyLinux
> (www.pokylinux.org) build system to pull from HG repos via tags.

If I understand you correctly, you have a whole bunch of different projects 
that you're hosting in one, huge, monolithic repository?  I find this 
interesting, because we have a similar use case where I work (several 
related, but non-interdependent, products that are released together), and we 
have a separate repository for each product.  *And* we're using a centralized 
VCS, not a distributed one like Mercurial.

I would have thought that you'd have one master repository per project, and 
then join them together via forest, or something (externals, in Subversion).  
If that were the case, developers would just clone whatever project they were 
currently working on, and only release management would have to worry about 
putting everything together into one monolithic release.  And developers 
could clone their forest repo, if it came to that.

> > the software. These maintainers will collects (using 'pull') the
> > changesets that affects their territories. Then, a global maintainer
> > will synchronize only with the components' maintainers.
>
> I don't believe that is realistic with developers working on many
> different and varying projects.  In our case, no one 'owns' particular

I agree; that's really non-scalable.  Of course, stuffing a whole bunch of 
different projects into a single repository isn't scalable, either; I don't 
know of a VCS that handles that very well.

> > Finally, the "frequent commit" policy is probably a legacy of the
> > locking model revision systems. Using mercurial, you will typically find
> > it much better to commit larger block of changes, and synchronizing your
> > repository even less frequently, for example when you more-or-less
> > completely finished implementing a new feature.
>
> We do frequent commits because we push out many products a year and
> have many many projects in development requiring frequent changes and
> releases.

Again, I agree.  However, there is a model of development (which derives from 
ClearCase, which by itself invalidates the model) where every feature is done 
in a branch and release management then cherry picks features that are 
supposed to go into a release.  I've never seen this work properly, mostly 
because it's based on a bunch of crack-smoking pipe dreams about release 
managers being able to merge code correctly; the point is that huge 
deliveries of code suck, and it is much, much easier to perform many smaller, 
incremental deliveries of code.

In most cases, though, the main argument is that the ClearCase model is more 
of a waterfall method, where iterative development is much more agile, and in 
software, agile almost always beats out waterfall in a cost/benefit analysis.

> >> For us, the push model is the only way.
> >
> > I hardly believe this.

I can easily believe that.  In fact, I don't believe that in any non-trivial 
project that the pull model is very effective; it just doesn't scale.  It 
introduces an artificial bottleneck -- the merge manager.  Especially not in 
a non-heterogeneous product.

> The pull model is just incredibly impractical at the enterprise level
> for embedded Linux product development, in my experience.

I'd say in almost any enterprise with a substantial code base.

--- SER