[Bug 3679] New: hgweb.cgi very slow - opens changelog tens of thousands of times

bugzilla-daemon at bz.selenic.com bugzilla-daemon at bz.selenic.com
Tue Oct 23 20:46:46 CDT 2012


http://bz.selenic.com/show_bug.cgi?id=3679

          Priority: normal
            Bug ID: 3679
                CC: mercurial-devel at selenic.com
          Assignee: bugzilla at selenic.com
           Summary: hgweb.cgi very slow - opens  changelog tens of
                    thousands of times
          Severity: bug
    Classification: Unclassified
                OS: Linux
          Reporter: afaigon at couponsinc.com
          Hardware: PC
            Status: UNCONFIRMED
           Version: unspecified
         Component: hgweb
           Product: Mercurial

We have a medium-size repository with a changelog of a bit over 13,000 commits.

Displaying the most recent changes over the web typically takes about 20
seconds, so users are complaining that "mercurial is slow"

Other repositories, with a small number of commits take typically < 2 seconds
to look at, which is great.

I straced the python process while it exectuted and found that over 90% of the
system call time is spent in open(2).

Here's the 'strace -c' summary:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 91.80    0.009063           0     99389         4 open
  5.14    0.000507           0    205765           read
  1.37    0.000135           0     99385           munmap
  0.86    0.000085           0    198771           fstat
  0.44    0.000043           0     99385           mmap
  0.24    0.000024           0     99385           close
  0.16    0.000016           0     99386           lseek
  0.00    0.000000           0         8           write
  0.00    0.000000           0         1         1 stat
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         1           rt_sigaction
------ ----------- ----------- --------- --------- ----------------
100.00    0.009873                901479         5 total

Drilling down to a detailed strace by system call duration (strace -T) doesn't
show any really specific system call as taking too long, so I suspect, not
sure, a very large number of context switches due to some system calls.  The
vast majority of open() calls are done on one file:

     /home/hg/[repository_name]/.hg/store/00changelog.d

So we have ~13,000 changes, only the latest few of them are showing on the
page, yet the changelog is being opened almost 100,000 times (~7 times per
change) during one call to hgweb.cgi.

I believe this may explain some of the slowness.
Can someone look into it?

Thanks!

-- 
You are receiving this mail because:
You are on the CC list for the bug.


More information about the Mercurial-devel mailing list