Bug 5960 - AttributeError: 'NoneType' object has no attribute 'startswith' when using inmemory merge for metadata-only mutations
Summary: AttributeError: 'NoneType' object has no attribute 'startswith' when using in...
Status: RESOLVED FIXED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: rebase (show other bugs)
Version: unspecified
Hardware: PC Linux
: wish feature
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-14 21:30 UTC by Kyle Lippincott
Modified: 2018-08-25 00:00 UTC (History)
3 users (show)

See Also:
Python Version: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kyle Lippincott 2018-08-14 21:30 UTC
With rebase.experimental.inmemory, changing a file from +x to -x or vice versa, with no content changes, can produce an exception.  This can be reproduced by adding the following to the bottom of test-rebase-inmemory.t:

Test a metadata-only in-memory merge
  $ cd $TEST_TMPDIR
  $ hg init no_exception
  $ cd no_exception
# Produce the following graph:
#   o  'add +x to foo.txt'
#   | o  r1  (adds bar.txt, just for something to rebase to)
#   |/
#   o  r0   (adds foo.txt, no +x)
  $ echo hi > foo.txt
  $ hg ci -qAm r0
  $ echo hi > bar.txt
  $ hg ci -qAm r1
  $ hg co -qr .^
  $ chmod +x foo.txt
  $ hg ci -qAm 'add +x to foo.txt'
BUG: this should not raise AttributeError. This test passes if the bug is PRESENT.
  $ hg rebase -r . -d 1 2>&1 | grep '^AttributeError'
  AttributeError: 'NoneType' object has no attribute 'startswith'


Reproduced with remotefilelog and without, and on several different versions of Mercurial.  What appears to be happening is that we call mercurial.context.overlayworkingctx.setflags for the path, which calls         self._markdirty(path, exists=True, date=dateutil.makedate(),
                        flags=(l and 'l' or '') + (x and 'x' or ''))

and self._markdirty defaults data=None, which then gets returned by fctx.data(), and things aren't prepared to handle this. I suspect the fix is going to be to make it so that setflags still sets the data, somehow, but I don't know nearly enough about this code to do this correctly.
Comment 1 Pulkit Goyal 2018-08-15 07:51 UTC
I encountered the same error message yesterday while hacking up on in-memory merge support for `hg merge` command.

I tried fixing this AttributeError by checking for None before getting into startswith, but there are other places in code which does not like fctx.data() as None. I think we should default the data to '' in self._markdirty or check for data before returning and set it to '' if we are returning None.
Comment 2 Kyle Lippincott 2018-08-15 19:55 UTC
I'm not sure that's safe to do - I suspect most of these codepaths assume that fctx.data() returns the actual contents, and not some placeholder (they may not crash if the placeholder is '', but I suspect they'll still do something incorrectly).

I think it makes sense for .data() to return the actual file contents, that seems to be what happens when non-in-memory-merge is used, but I don't know how it gets access to these contents normally.
Comment 3 Phil Cohen 2018-08-15 20:19 UTC
I think it should be safe to "page in" the underlying's context data and flags into the overlayworkingctx's cache during `_markdirty` if it wasn't provided. 

I'm not sure why I didn't do it this way initially to be honest.
Comment 4 HG Bot 2018-08-17 21:01 UTC
Fixed by https://mercurial-scm.org/repo/hg/rev/95bd19f60957
Kyle Lippincott <spectral@google.com>
overlayworkingctx: fix exception in metadata-only inmemory merges (issue5960)

If there was a metadata-only mutation, such as +x or -x on a file, we would
create a cache entry with None for data, and this would cause problems later on
when some code tried to run fctx.data() or similar, and was expecting a string.

My original fix for this involved passing data=self._wrappedctx[path].data() in
setflags(), but this version seems slightly better - this way, if we ever call
write() and then call setflags(), we don't destroy the data that we wrote that's
in the cache. I haven't verified that other fields aren't destroyed, such as
date or flags :)

Differential Revision: https://phab.mercurial-scm.org/D4287

(please test the fix)
Comment 5 Bugzilla 2018-08-25 00:00 UTC
Bug was set to TESTING for 7 days, resolving