RFC: exact change detection for non append-only files

[Pierre-Yves David]

On 11/18/2015 05:02 AM, FUJIWARA Katsunori wrote:
>
> At Tue, 17 Nov 2015 18:49:38 -0600,
> Matt Mackall wrote:
>>
>> On Wed, 2015-11-18 at 02:10 +0900, FUJIWARA Katsunori wrote:
>>> Now, 'filecache' detects changes of files by 'cachestat.__eq__()' of
>>> posix.py on POSIX platform, and it examines:
>>>
>>>    - st_size:
>>>
>>>      This works for append-only files (like revlog) as expect in all
>>>      cases (doesn't it ?)
>>>
>>>      But some status files (e.g. dirstate, bookmarks and so on) may
>>> not
>>>      be changed in size, even if they are actually changed.
>>>
>>>    - st_mtime:
>>>
>>>      For non append-only files, this works as expect in many cases.
>>> But
>>>      'st_mtime' doesn't have enough resolution for recent computing
>>> and
>>>      I/O speed, even if it is represented in float (see also issue4836
>>>      for more detail).
>>>
>>>    - st_ino:
>>>
>>>      This can compensate for 'st_mtime', because copy-on-write
>>>      semantics always changes st_ino.
>>>
>>> Therefore, 'st_ino' is the last bastion for change detection of
>>> dirstate and so on.
>>>
>>> But inode is quickly reused on some filesystems (perhaps for
>>> performance reason), and it prevents examination of 'st_ino' from
>>> detecting changes as expected.
>>>
>>> My instant ideas to detect changes correctly even in such situation
>>> are:
>>>
>>>    - ignore this very very rare case :-)
>>
>> Has it ever been observed?
>>
>> We'd probably need the following sequence of events to trigger it:
>>
>> - create file with stats (S1, M1, I1)
>> - replace with file (S1, M1, I2)
>> - replace again with file (S1, M1, I1) in the same second
>>
>> We also have st_ctime in the tuple.
>
> I found this issue while checking for issue4368, and this assumes
> parallel processing.
>
>      https://bz.mercurial-scm.org/show_bug.cgi?id=4368
>
> At parallel commits, invalidation of 'repo.dirstate' at acquisition of
> wlock fails to discard already loaded dirstate data very very rarely.

Can you update the bug with this data?

-- 
Pierre-Yves David