[PATCH 1 of 3] util: add fswatcher class

Matt Mackall mpm at selenic.com
Mon May 30 11:07:37 CDT 2011


On Fri, 2011-05-27 at 23:38 +0200, Sune Foldager wrote:
> On 2011-05-27 11:55, Matt Mackall wrote:
> >On Fri, 2011-05-27 at 16:18 +0200, Sune Foldager wrote:
> >> On 2011-05-27 07:40, Matt Mackall wrote:
> >> >On Fri, 2011-05-27 at 12:22 +0200, Sune Foldager wrote:
> >> >> >
> >> >> >Not at all sure about this mtime/ctime comparison business you're doing.
> >> >> >Not only is its safety suspect, several filesystems don't have the
> >> >> >concept of a ctime.
> >> >>
> >> >> Well, the ctime is from time.time() to not really related to the file system.
> >> >
> >> >Ahh, I see. Please, pick another variable name, it's impossible for a
> >> >Unix hacker to see mtime and ctime on the same line and not think
> >> >they're both referring to stat fields.
> >>
> >> Alright.
> >>
> >> >> Benoit told me this was the usual way to sidestep the sub-second problem.
> >> >
> >> >Well, no, not exactly. We can't actually trust that system notion of
> >> >time is connected to the filesystem's notion of time (see network
> >> >filesystem). If all parties don't have a reliable NTP setup, it's quite
> >> >easy for causality to appear to be violated. Dirstate does:
> >> >
> >> >        # use the modification time of the newly created temporary file as the
> >> >        # filesystem's notion of 'now'
> >> >        now = int(util.fstat(st).st_mtime)
> >> >
> >> >For our watcher purposes, I'm not sure if there's a good way to do this
> >> >short of writing a temp file (to the same fs).
> >>
> >> Somewhat annoying to have to do that, and where should we write to, etc.
> >> It requires more knowledge than the wather has currently.
> >
> >The safest choice is in the same directory as the watched file.
> >And yes, highly annoying.
> >
> >But can be skipped entirely if we've got subsecond timestamps.
> 
> >[...]
> 
> >We're only interested in 'realistic cases'. People with write access who
> >want to break things don't need to trick us to do it, so we needn't
> >concern ourselves with anything that wouldn't happen in normal use.
> 
> Agreed.
> 
> Alright. To move forward, can we put the fswatcher in (maybe with size check
> as well, or something?) and write a localrepo-method as you suggested?
> Then hgweb can use it, and will at least do better than it does now. We can
> always improve this later on, or even right after it's in :p.

Yes.

> As for subsecond timestamps, sure. os.stat doesn't seem to give them, though.

On firefly, running ext4:

$ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.stat_float_times()
True
>>> s = os.stat('.')
>>> s
posix.stat_result(st_mode=16877, st_ino=533201, st_dev=2049L,
st_nlink=6, st_uid=1005, st_gid=1005, st_size=4096, st_atime=1306351490,
st_mtime=1306350746, st_ctime=1306350746)
>>> s.st_mtime
1306350746.4471037

On my laptop, running ext3:

$ python
Python 2.6.6 (r266:84292, Apr 20 2011, 09:34:38) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.stat_float_times()
True
>>> s = os.stat('.')
>>> s
posix.stat_result(st_mode=16877, st_ino=3604181, st_dev=2049L,
st_nlink=188, st_uid=1000, st_gid=1000, st_size=20480,
st_atime=1306726652, st_mtime=1306438677, st_ctime=1306438677)
>>> s.st_mtime
1306438677.0
>>> 

Various efforts have been made over the years to squeeze nanonseconds
into ext3, but as it uses an inode format inherited from ext2, there
just wasn't enough room.

But we can expect newly-installed Linux systems to use ext4 or btrfs
from now on. NTFS, XFS, JFS, UFS, and a bunch of others also support
nanoseconds. 

HFS+ apparently only has second resolution. And VFAT and friends only
have 2-second resolution. So we're going to have to hold on to
sub-second hacks for a while.

> >>  Maybe we should, for the purpose of hgweb and
> >> hgwebdir, just accept that we can't catch all cases and then do one of
> >> two things:
> >>
> >> - Do pretty well, e.g. maybe not start writing files to get a concept of
> >>    current time, unless we feel it's really needed. And maybe or maybe not
> >>    comparing sizes etc. We won't catch all cases, but some.
> >> - Document that it's peoples own responsibility to make sure the repo isn't
> >>    modified under their noses while the hold a repo instance, and then not
> >>    really check for it at all.
> >
> >Maybe. Maybe it's also time to introduce the reader lock I talked about
> >earlier:
> >
> >http://markmail.org/message/2kz57imteeu5tqmx
> >
> >If we think of things in terms of readers, appenders, and destroyers
> >rather than readers and writers, we can see that we actually don't care
> >all that much about appenders. That is, so long as people aren't
> >actually deleting things, the view exported by hgweb is still 'causally
> >plausible' and valid, even if it gets a little behind.
> 
> I agree, which is also why I think we should move forward a bit :).

-- 
Mathematics is the supreme nostalgia of our time.




More information about the Mercurial-devel mailing list