D7144: status: use unfiltered repo if we're getting status of working copy
martinvonz (Martin von Zweigbergk)
phabricator at mercurial-scm.org
Wed Oct 30 16:48:44 EDT 2019
martinvonz added a comment.
In D7144#105248 <https://phab.mercurial-scm.org/D7144#105248>, @mharbison72 wrote:
> The comment about maybe affecting extensions made me think of df463ca0adef <https://phab.mercurial-scm.org/rHGdf463ca0adefbf6ffa084cfd8031dcfc6c944f8e>. I never figured out what the issue was.
I suspect that's because largefiles creates a repo subclass and that subclass does not have the `__getattr__` and `__setattr__` overrides that `repoview` does, so setting `lfstatus`or `_largefilesenabled` on that subclass won't propagate it to the parent.
In D7144#105352 <https://phab.mercurial-scm.org/D7144#105352>, @marmoute wrote:
> In D7144#105048 <https://phab.mercurial-scm.org/D7144#105048>, @martinvonz wrote:
>
>> In D7144#105039 <https://phab.mercurial-scm.org/D7144#105039>, @marmoute wrote:
>>
>>> I have a couple of questions:
>>>
>>> […]
>>>
>>> - This could cause issue to extensions, can we abtract the unfiltering though a function so that extension can rewrap it ?
>>
>> To make sure I address it properly, what's the issue(s) you're thinking of?
>
> Imagine you are an extension writer adding extra information on status (eg: tell the user if his stack is behind and he needs to rebase). If you get an unfiltered repository, any operation you run (eg: a revset to check the graph, will go horribly wrong). So we need need at minimum an easy way to disable the unfiltering if an extension needs to do it.
>
> In D7144#105066 <https://phab.mercurial-scm.org/D7144#105066>, @martinvonz wrote:
>
>> In D7144#105065 <https://phab.mercurial-scm.org/D7144#105065>, @marmoute wrote:
>>
>>> Another question is " Why are we loading the changelog if we don't need it ?". If the code were not accessing the changelog, we could not pay the filtering code (and save other parsing cost).
>>
>> We load the changelog when we do `repo['.']`. I had added a hack (https://www.mercurial-scm.org/repo/hg/file/4565a0afc289/mercurial/localrepo.py#l1539) to try to avoid that, but it was no longer effective. I think using the unfiltered repo is a better solution anyway.
>
> If the performance gain are clear, using an unfiltered repository seems like a reasonable gain/risk move. However I would prefer this to remain temporary. Keeping the repository filtering aligned to the current semantic is important to avoid surprise and obscure bug in the future. We need an overall effort to reduce the filtering impact anyway (independantly from this series).
> In the case here, I wonder if we have and easy way to prevent the filtering computation kick in, as it is clearly unnecessary.
>
> - can you share a profile of what takes time with the filtering version
> - Do you know which part is triggering the filtering ?
I'll get back to you about the other things later, but I think the following profile answers these last questions:
| 100.0% 0.19s dispatch.py: _callcatch line 36: dispatch.run()
| 100.0% 0.19s scmutil.py: callcatch line 433: return scmutil.callcatch(ui...
| 100.0% 0.19s dispatch.py: _runcatchfunc line 177: return func()
| 100.0% 0.19s hg.py: repository line 414: return _dispatch(req)
\ 52.6% 0.10s localrepo.py: status line 6831: opts.get(b'subrepos'),
\ 31.6% 0.06s localrepo.py: __getitem__ line 3248: return self[node1].status(
| 31.6% 0.06s repoview.py: changelog line 1547: rev = self.changelog.rev(ch...
| 31.6% 0.06s repoview.py: filterrevs line 278: revs = filterrevs(unfi, sel...
| 31.6% 0.06s repoview.py: computehidden line 217: repo.filteredrevcache[filte...
\ 15.8% 0.03s repoview.py: hideablerevsline 87: hidden = hideablerevs(repo)
| 15.8% 0.03s obsolete.py: getrevs line 39: obsoletes = obsolete.getrev...
| 15.8% 0.03s obscache.py: <lambda> line 905: repo.obsstore.caches[name] ...
| 15.8% 0.03s obscache.py: _computeobsoletesetline 469: wrapped = lambda repo: _com...
| 10.5% 0.02s phases.py: getrevset line 434: notpublic = repo._phasecach...
\ 5.3% 0.01s scmutil.py: __get__ line 106: return super(_basefilecache...
| 5.3% 0.01s localrepo.py: _phasecacheline 1636: entry.obj = self.func(obj)
| 5.3% 0.01s phases.py: __init__line 1432: return phases.phasecache(se...
| 5.3% 0.01s phases.py: _readrootsline 235: self.phaseroots, self.dirty...
\ 5.3% 0.01s phases.py: loadphaserevsline 243: self.loadphaserevs(repo) #...
| 5.3% 0.01s phases.py: _getphaserevsnativeline 325: res = self._getphaserevsnat...
| 5.3% 0.01s changelog.py: rev line 301: pycompat.maplist(repo.chang...
\ 10.5% 0.02s repoview.py: pinnedrevs line 89: hidden = set(hidden - pinne...
\ 5.3% 0.01s repoview.py: <genexpr> line 58: pinned.update(rev(t[0]) for...
\ 5.3% 0.01s tags.py: readlocaltagsline 55: tagsmod.readlocaltags(repo....
| 5.3% 0.01s tags.py: _readtagsline 255: ui, repo, data.splitlines()...
| 5.3% 0.01s tags.py: _readtaghistline 337: ui, repo, lines, fn, recode...
\ 5.3% 0.01s repoview.py: _revealancestorsline 96: _revealancestors(pfunc, hid...
| 5.3% 0.01s smartset.py: _iterfilterline 73: stack = list(revs)
| 5.3% 0.01s smartset.py: __init__ line 467: l = baseset(r for r in self)
| 5.3% 0.01s smartset.py: <genexpr> line 242: data = list(data)
| 5.3% 0.01s smartset.py: _iterfilterline 467: l = baseset(r for r in self)
| 5.3% 0.01s smartset.py: <lambda> line 421: if cond(x):
\ 21.1% 0.04s context.py: status line 3249: node2, match, ignored, clea...
| 21.1% 0.04s localrepo.py: narrowmatch line 424: ctx1, r, match, listignored...
| 21.1% 0.04s context.py: _dirstatestatusline 1878: s = self._dirstatestatus(ma...
| 21.1% 0.04s dirstate.py: status line 1788: match, subrepos, ignored=ig...
| 21.1% 0.04s dirstate.py: walk line 1117: self.walk(match, subrepos, ...
\ 15.8% 0.03s dirstate.py: traverse line 1018: traverse([d], alreadynormed)
| 5.3% 0.01s dirstate.py: __contains__line 1001: if nf in dmap:
\ 5.3% 0.01s scmutil.py: __get__ line 911: ignore = self._ignore
| 5.3% 0.01s dirstate.py: _ignore line 1636: entry.obj = self.func(obj)
| 5.3% 0.01s match.py: match line 167: return matchmod.match(self....
| 5.3% 0.01s match.py: _buildkindpatsmatcherline 277: badfn=None,
| 5.3% 0.01s match.py: __init__ line 127: m = matchercls(root, kindpa...
| 5.3% 0.01s match.py: _buildmatchline 673: self._pats, self.matchfn = ...
| 5.3% 0.01s match.py: _buildregexmatchline 1368: regex, mf = _buildregexmatc...
| 5.3% 0.01s match.py: _rematcherline 1417: matcher = _rematcher(fullre...
| 5.3% 0.01s util.py: compile line 52: m = util.re.compile(regex)
| 5.3% 0.01s re.py: compile line 2168: return remod.compile(pat, f...
| 5.3% 0.01s re.py: _compile line 194: return _compile(pattern, fl...
| 5.3% 0.01s sre_compile.py: compile line 249: p = sre_compile.compile(pat...
| 5.3% 0.01s sre_parse.py: parse line 572: p = sre_parse.parse(p, flags)
| 5.3% 0.01s sre_parse.py: _parse_subline 735: p = _parse_sub(source, patt...
| 5.3% 0.01s sre_parse.py: _parse line 343: itemsappend(_parse(source, ...
| 5.3% 0.01s sre_parse.py: append line 448: subpatternappend((LITERAL, ...
\ 21.1% 0.04s ui.py: pager line 6843: ui.pager(b'status')
| 21.1% 0.04s ui.py: _runpager line 1289: if self._runpager(pagercmd,...
| 21.1% 0.04s subprocess.py: __init__ line 1347: env=procutil.tonativeenv(pr...
| 21.1% 0.04s subprocess.py: _execute_child line 394: errread, errwrite)
| 21.1% 0.04s subprocess.py: _eintr_retry_callline 1023: data = _eintr_retry_call(os...
\ 15.8% 0.03s extensions.py: load line 294: load(ui, name, path, loadin...
| 15.8% 0.03s extensions.py: _importext line 211: mod = _importext(name, path...
\ 10.5% 0.02s extensions.py: _importh line 127: mod = _importh(b"hgext3rd.%...
| 10.5% 0.02s demandimportpy2.py: _demandimportline 107: mod = __import__(pycompat.s...
| 10.5% 0.02s demandimportpy2.py: _hgextimportline 181: return _hgextimport(_origim...
| 10.5% 0.02s __init__.py: <module> line 44: return importfunc(name, glo...
\ 5.3% 0.01s demandimportpy2.py: __getattr__line 344: eh.merge(obsexchange.eh)
| 5.3% 0.01s demandimportpy2.py: _loadline 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s obsexchange.py: <module>line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__line 36: eh.merge(obsdiscovery.eh)
| 5.3% 0.01s demandimportpy2.py: _loadline 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s obsdiscovery.py: <module>line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__line 332: class _obshashcache(obscach...
| 5.3% 0.01s demandimportpy2.py: _loadline 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s obscache.py: <module>line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__line 28: obsstorefilecache = localre...
| 5.3% 0.01s demandimportpy2.py: _loadline 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s localrepo.py: <module>line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: _demandimportline 30: from . import (
| 5.3% 0.01s demandimportpy2.py: processfromitemline 278: processfromitem(mod, x)
\ 5.3% 0.01s cmdutil.py: findcmd line 332: aliases, entry = cmdutil.fi...
| 5.3% 0.01s cmdutil.py: findpossibleline 859: choice, allcmds = findpossi...
\ 5.3% 0.01s extensions.py: _importh line 122: mod = _importh(b"hgext.%s" ...
| 5.3% 0.01s demandimportpy2.py: _demandimportline 107: mod = __import__(pycompat.s...
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 181: return _hgextimport(_origim...
| 5.3% 0.01s phabricator.py: <module> line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__line 51: help.CATEGORY_ORDER.insert(
| 5.3% 0.01s demandimportpy2.py: _load line 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s help.py: <module> line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__line 592: addtopicsymbols(b'filesets'...
| 5.3% 0.01s demandimportpy2.py: _load line 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s fileset.py: <module> line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: _demandimportline 15: from . import (
| 5.3% 0.01s demandimportpy2.py: processfromitemline 278: processfromitem(mod, x)
\ 5.3% 0.01s scmutil.py: revpair line 6782: ctx1, ctx2 = scmutil.revpai...
| 5.3% 0.01s localrepo.py: __getitem__ line 732: return repo[b'.'], repo[None]
| 5.3% 0.01s dirstate.py: p1 line 1542: node = self.dirstate.p1()
| 5.3% 0.01s localrepo.py: _dirstatevalidateline 293: return self._validate(self....
| 5.3% 0.01s localrepo.py: __get__ line 1460: self.changelog.rev(node)
| 5.3% 0.01s scmutil.py: __get__ line 106: return super(_basefilecache...
| 5.3% 0.01s localrepo.py: changelog line 1636: entry.obj = self.func(obj)
| 5.3% 0.01s store.py: changelog line 1440: return self.store.changelog...
| 5.3% 0.01s changelog.py: __init__ line 424: return changelog.changelog(...
| 5.3% 0.01s demandimportpy2.py: _load line 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s changelog.py: <module> line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: __getattr__ line 354: class changelog(revlog.revl...
| 5.3% 0.01s demandimportpy2.py: _load line 157: self._load()
| 5.3% 0.01s demandimportpy2.py: _hgextimportline 97: _origimport, head, globals,...
| 5.3% 0.01s revlog.py: <module> line 44: return importfunc(name, glo...
| 5.3% 0.01s demandimportpy2.py: _demandimportline 39: from .revlogutils.constants...
| 5.3% 0.01s demandimportpy2.py: processfromitemline 278: processfromitem(mod, x)
\ 5.3% 0.01s fancyopts.py: fancyopts line 922: args = fancyopts.fancyopts(...
---
Sample count: 103
Total time: 0.180000 seconds (0.190000 wall)
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST ACTION
https://phab.mercurial-scm.org/D7144/new/
REVISION DETAIL
https://phab.mercurial-scm.org/D7144
To: martinvonz, #hg-reviewers
Cc: mharbison72, pulkit, marmoute, mercurial-devel
More information about the Mercurial-devel
mailing list