[issue1752] fetch much slower in 1.3

[Jesse Glick]

New submission from Jesse Glick <jesse.glick at sun.com>:

Running 'hg fetch' (when it needs to actually commit a merge) seems to be
far slower in Hg 1.3 than in previous releases.

For example, when using http://hg.netbeans.org/main-silver/ clones
(containing ~93k files), it is typical to make a few commits touching a few
files, then 'hg fetch' to update others' changes and merge your commits
against the new remote head. In previous releases, the commit operation -
time between e.g. "5 files updated, 0 files merged, 0 files removed, 0 files
unresolved" and "new changeset ... merges remote changes with local" - would
take maybe a second or two. Now it takes around ten minutes!

I was not able to get a Python stack trace, but strace showed Hg opening
every file in the store and in the working copy, e.g.

lstat64("/space/src/nb/core-main/visualweb.dataconnectivity/src/org/netbeans/modules/visualweb/dataconnectivity/resources/started.png",
{st_mode=S_IFREG|0644, st_size=229, ...}) = 0
open("/space/src/nb/core-main/visualweb.dataconnectivity/src/org/netbeans/modules/visualweb/dataconnectivity/resources/started.png",
O_RDONLY|O_LARGEFILE) = 7
fstat64(7, {st_mode=S_IFREG|0644, st_size=229, ...}) = 0
fstat64(7, {st_mode=S_IFREG|0644, st_size=229, ...}) = 0
_llseek(7, 0, [0], SEEK_CUR)            = 0
fstat64(7, {st_mode=S_IFREG|0644, st_size=229, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7ab4000
_llseek(7, 0, [0], SEEK_CUR)            = 0
read(7,
"\211PNG\r\n\32\n\0\0\0\rIHDR\0\0\0\t\0\0\0\t\10\6\0\0\0\340\221\6\20"...,
4096) = 229
read(7, ""..., 4096)                    = 0
close(7)                                = 0
munmap(0xb7ab4000, 4096)                = 0
open("/space/src/nb/core-main/.hg/store/data/visualweb.dataconnectivity/src/org/netbeans/modules/visualweb/dataconnectivity/resources/started.png.i",
O_RDONLY|O_LARGEFILE) = 7
fstat64(7, {st_mode=S_IFREG|0644, st_size=289, ...}) = 0
mmap2(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
= 0xb79b4000
fstat64(7, {st_mode=S_IFREG|0644, st_size=289, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb79b3000
read(7,
"\0\1\0\1\0\0\0\0\0\0\0\341\0\0\0\345\0\0\0\0\0\0\375Z\377\377\377\377\377\377\377\377W"...,
1048576) = 289
read(7, ""..., 1044480)                 = 0
mremap(0xb79b4000, 1052672, 4096, MREMAP_MAYMOVE) = 0xb79b4000
close(7)                                = 0
munmap(0xb79b3000, 4096)                = 0
munmap(0xb79b4000, 4096)                = 0
lstat64("/space/src/nb/core-main/visualweb.dataconnectivity/src/org/netbeans/modules/visualweb/dataconnectivity/resources/started.png",
{st_mode=S_IFREG|0644, st_size=229, ...}) = 0
lstat64("/space/src/nb/core-main/visualweb.dataconnectivity/src/org/netbeans/modules/visualweb/dataconnectivity/resources/started.png",
{st_mode=S_IFREG|0644, st_size=229, ...}) = 0

I suspect 4ea995e5aac0 ("fetch: no need to pass files list to commit") and
25e9c71b89de ("commit: drop the now-unused files parameter") is the culprit.

----------
assignedto: mpm
messages: 10134
nosy: jglick, mpm
priority: urgent
status: unread
title: fetch much slower in 1.3
topic: performance, regression

____________________________________________________
Mercurial issue tracker <mercurial-bugs at selenic.com>
<http://mercurial.selenic.com/bts/issue1752>
____________________________________________________