Bug 5716 - severe perf regression in hg update because of path conflict checking code
Summary: severe perf regression in hg update because of path conflict checking code
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: 4.4-rc
Hardware: All All
: urgent bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-10-23 19:12 UTC by Siddharth Agarwal
Modified: 2018-01-19 15:11 UTC (History)
3 users (show)

See Also:
Python Version: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Siddharth Agarwal 2017-10-23 19:12 UTC
We're tracking a severe performance regression in hg update in 4.4rc that appears to be because of path conflict checking code.
Comment 1 Siddharth Agarwal 2017-10-23 19:12 UTC
+Greg for possible impact to Mozilla
Comment 2 Boris Feld 2017-10-24 07:54 UTC
Do you have more details you could share? Does it happens when lots of files are updated? What is the performance gap detected?
Comment 3 HG Bot 2017-11-01 16:20 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/2a774cae3a03
Siddharth Agarwal <sid0@fb.com>
merge: disable path conflict checking by default (issue5716)

We shouldn't ship a severe perf regression in hg update for 4.4.

Differential Revision: https://phab.mercurial-scm.org/D1223

(please test the fix)
Comment 4 Bugzilla 2017-11-09 00:00 UTC
Bug was set to TESTING for 7 days, resolving
Comment 5 HG Bot 2017-12-01 15:10 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/b85962350bb3
Mark Thomas <mbthomas@fb.com>
merge: cache unknown dir checks (issue5716)

As mentioned in D1222, the recent pathconflicts change regresses update
performance in large repositories when many files are being updated.

To mitigate this, we introduce two caches of directories that have
already found to be either:

  - unknown directories, but which are not aliased by files and
    so don't need to be checked if they are files again; and

  - missing directores, which cannot cause path conflicts, and
    cannot contain a file that causes a path conflict.

When checking the paths of a file, testing against this caches means we can
skip tests that involve touching the filesystem.

Differential Revision: https://phab.mercurial-scm.org/D1224

(please test the fix)
Comment 6 HG Bot 2017-12-01 15:10 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/a92b9f8e11ba
Mark Thomas <mbthomas@fb.com>
merge: check created file dirs for path conflicts only once (issue5716)

In large repositories, updates involving the creation of many files check the
same directories repeatedly in the wctx manifest.  Move these checks out to a
separate loop to avoid repeated checks hitting the manifest.

Differential Revision: https://phab.mercurial-scm.org/D1226

(please test the fix)
Comment 7 Bugzilla 2017-12-09 00:00 UTC
Bug was set to TESTING for 7 days, resolving
Comment 8 Augie Fackler 2018-01-19 15:11 UTC
*** Bug 5728 has been marked as a duplicate of this bug. ***