[Bug 4756] New: hg serve: revlog parsing and integrity checks fail under load

[mercurial-bugs at selenic.com]

http://bz.selenic.com/show_bug.cgi?id=4756

          Priority: normal
            Bug ID: 4756
                CC: mercurial-devel at selenic.com
          Assignee: bugzilla at selenic.com
           Summary: hg serve: revlog parsing and integrity checks fail
                    under load
          Severity: bug
    Classification: Unclassified
                OS: Linux
          Reporter: av6 at dwimlabs.net
          Hardware: PC
            Status: UNCONFIRMED
           Version: default branch
         Component: hgweb
           Product: Mercurial

Set up:

export HGPLAIN=1
export HGRCPATH=/dev/null
hg clone https://selenic.com/hg hg
make -C hg local
./hg/hg verify -R hg
./hg/hg serve -R hg -E error.log

Simulating load:

siege -b http://127.0.0.1:8000/tags

hg serve works as expected, but after a couple of seconds error.log contains
two kinds of errors:

127.0.0.1 - - [18/Jul/2015 19:15:12] Exception happened during processing
request '/tags':
Traceback (most recent call last):
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 80, in do_POST
    self.do_write()
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 73, in do_write
    self.do_hgweb()
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 141, in do_hgweb
    for chunk in self.server.application(env, self._start_response):
  File "/tmp/testfailtags/hg/mercurial/util.py", line 613, in increasingchunks
    for chunk in source:
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 714, in _flatten
    for j in _flatten(i):
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 707, in _flatten
    for i in thing:
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 279, in runmap
    for i in d:
  File "/tmp/testfailtags/hg/mercurial/hgweb/webcommands.py", line 588, in
entries
    "date": web.repo[n].date(),
  File "/tmp/testfailtags/hg/mercurial/context.py", line 523, in date
    return self._changeset[2]
  File "/tmp/testfailtags/hg/mercurial/util.py", line 531, in __get__
    result = self.func(obj)
  File "/tmp/testfailtags/hg/mercurial/context.py", line 498, in _changeset
    return self._repo.changelog.read(self.rev())
  File "/tmp/testfailtags/hg/mercurial/changelog.py", line 323, in read
    text = self.revision(node)
  File "/tmp/testfailtags/hg/mercurial/revlog.py", line 1094, in revision
    bins = self._chunks(chain)
  File "/tmp/testfailtags/hg/mercurial/revlog.py", line 1015, in _chunks
    ladd(decompress(buffer(data, chunkstart - offset, chunklength)))
ValueError: offset must be zero or positive

and

127.0.0.1 - - [18/Jul/2015 19:15:20] Exception happened during processing
request '/tags':
Traceback (most recent call last):
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 80, in do_POST
    self.do_write()
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 73, in do_write
    self.do_hgweb()
  File "/tmp/testfailtags/hg/mercurial/hgweb/server.py", line 141, in do_hgweb
    for chunk in self.server.application(env, self._start_response):
  File "/tmp/testfailtags/hg/mercurial/util.py", line 613, in increasingchunks
    for chunk in source:
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 714, in _flatten
    for j in _flatten(i):
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 707, in _flatten
    for i in thing:
  File "/tmp/testfailtags/hg/mercurial/templater.py", line 279, in runmap
    for i in d:
  File "/tmp/testfailtags/hg/mercurial/hgweb/webcommands.py", line 588, in
entries
    "date": web.repo[n].date(),
  File "/tmp/testfailtags/hg/mercurial/context.py", line 523, in date
    return self._changeset[2]
  File "/tmp/testfailtags/hg/mercurial/util.py", line 531, in __get__
    result = self.func(obj)
  File "/tmp/testfailtags/hg/mercurial/context.py", line 498, in _changeset
    return self._repo.changelog.read(self.rev())
  File "/tmp/testfailtags/hg/mercurial/changelog.py", line 323, in read
    text = self.revision(node)
  File "/tmp/testfailtags/hg/mercurial/revlog.py", line 1101, in revision
    text = self._checkhash(text, node, rev)
  File "/tmp/testfailtags/hg/mercurial/revlog.py", line 1116, in _checkhash
    self.checkhash(text, p1, p2, node, rev)
  File "/tmp/testfailtags/hg/mercurial/revlog.py", line 1125, in checkhash
    % (self.indexfile, revornode))
RevlogError: integrity check failed on 00changelog.i:25805

(the number after 00changelog.i varies: 23246, 20341, 24892, 9984, etc)

It can be more or less reliably reproduced using /tags, but I couldn't get the
errors on /bookmarks or /file or /, for example.

Also, I couldn't reproduce this serving with --web-conf:

[paths]
/hg = ./hg

./hg/hg serve --web-conf=hgweb.conf -E error.log

siege -b http://127.0.0.1:8000/hg/tags

This doesn't throw any error.

This works fine too:

while true; do ./hg/hg tags -R hg > /dev/null; done

-- 
You are receiving this mail because:
You are on the CC list for the bug.