Performance issues when merging branches (1000+ files being merged takes 75~ seconds)

Idan K idan at kamarafamily.com
Thu Jun 17 08:40:31 CDT 2010


Running "hg merge --profile -q -y branch-a" yielded the attached table
(couldn't get the formatting right in the mail).

On Thu, Jun 17, 2010 at 3:43 PM, Greg Ward <greg-hg at gerg.ca> wrote:

> On Wed, Jun 16, 2010 at 5:46 PM, Idan K <idan at kamarafamily.com> wrote:
> > 1. Clone Mercurial's repository (http://hg.intevation.org/mercurial/),
> > updated to default.
> > 2. Create a named branch "branch-a".
> > 3. For every file in the repository, insert a new line after line #3 (if
> the
> > file contains 3 lines), containing "### ADDED TEXT ###".
> > 4. Commit.
> > 5. Update back to default.
> > 6. Create a named branch "branch-b".
> > 7. Repeat step 3, but for line #9.
> > 8. Commit.
> > 9. Merge "branch-a" to "branch-b".
> >
> > I ran this test on a Ubuntu 10.04 with Mercurial 1.5.4 and the result was
> > quite surprising to say the least: Step #9 took about 75 seconds (average
> of
> > 5 runs) on a Q6600 with 4GB RAM, ext4 drive.
> > I also ran the same test with Git (using fast-export to covert
> Mercurial's
> > repository to Git) to get an idea of what I should be getting, and this
> time
> > step #9 took an average of 1 second. That's quite a difference. I know
> > Mercurial is written in Python (with some C modules) and Git in C, but
> can
> > the gap be that big?
>
> Ouch.  I'm convinced that if you dig far enough back into git's
> history, you'll find a commit like
>
>  Author: Linus
>  Date: distant mists of time
>  Description:
>    sell soul to the devil for performance at any cost ... *evil laugh*
>
> Generally when there is a performance gap this big, it's not a Python
> vs. C thing, it's a case of Mercurial using an inefficient algorithm,
> or the wrong data structure, or not hitting a fast code path for some
> reason.  Very often a small pure-Python patch can make a dramatic
> performance improvement.
>
> The first step is to profile.  If you don't mind, can you re-run your
> slow merge with "hg --profile" and post the results?
>
> Thanks!
>
> Greg
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100617/d2080c24/attachment.htm>
-------------- next part --------------
   CallCount    Recursive    Total(ms)   Inline(ms) module:lineno(function)
        1084            0     93.4493     46.7744   mercurial.ancestor:10(ancestor)
   +23959652            0     39.4046     29.3012   +mercurial.revlog:1142(parents)
   +12320744            0      3.3415      3.3415   +<min>
   +12322912            0      2.2564      2.2564   +<method 'pop' of 'list' objects>
   +12320744            0      1.6366      1.6366   +<method 'append' of 'list' objects>
       +4336            0      0.0357      0.0121   +mercurial.ancestor:55(generations)
    23959652            0     39.4046     29.3012   mercurial.revlog:1142(parents)
   +23959652            0     10.1034     10.1034   +mercurial.revlog:523(parentrevs)
    23960738            0     10.1045     10.1045   mercurial.revlog:523(parentrevs)
    12326977            0      3.3441      3.3441   <min>
       20940            0      2.2836      2.2836   <open>
    12330763            0      2.2598      2.2598   <method 'pop' of 'list' objects>
        1084            0     95.7394      2.2525   mercurial.revlog:1133(ancestor)
       +1084            0     93.4493     46.7744   +mercurial.ancestor:10(ancestor)
       +2168            0      0.0081      0.0072   +mercurial.ancestor:55(generations)
       +1084            0      0.0050      0.0050   +<sorted>
       +1084            0      0.0190      0.0037   +mercurial.revlog:1125(descendant)
       +1084            0      0.0036      0.0036   +mercurial.revlog:515(node)
        2166            0      6.4694      2.1665   mercurial.merge:38(_write)
    +1762041            0      1.3365      1.3365   +<method 'write' of 'file' objects>
    +1759875            0      0.8898      0.8898   +<method 'join' of 'str' objects>
       +2166            0      2.0727      0.0132   +mercurial.util:833(__call__)
       +2166            0      0.0026      0.0026   +<binascii.hexlify>
       +2166            0      0.0013      0.0013   +<method 'iteritems' of 'dict' objects>
    12363556            0      1.6452      1.6452   <method 'append' of 'list' objects>
     2020396            0      1.5726      1.5726   <method 'write' of 'file' objects>


More information about the Mercurial-devel mailing list