OS X FSEvents and temporary files

[Matt Mackall]

I talked with Martin a bit about the FSEvents problem you're having with
MacHG and took a closer look at what Mercurial's doing and how Apple's
API works. In summary, it looks like this:

1. MacHG gets notified of a change in repo/
2. MacHG asks Mercurial for status info on repo/
3. Mercurial scans repo/ and encounters a file with the x bit changed
4. Mercurial attempts to detect whether this is a real change by
detecting whether the filesystem that repo/ is on actually supports the
x bit by creating a temp file and changing its mode
5. MacHG gets notified of another change in repo/
6. MacHG calls status

So there's a couple salient facts here:

a) the FSEvents API only reports changes at a directory granularity
b) it also aggregates events in directory trees
c) if directory events are ignored, detection of real changes may be
delayed indefinitely
d) Mercurial needs these tests to deal with non-Unix-like filesystems,
which may be present on Macs
e) it appears the temp files only get created/destroyed when there are
files with the exec or symlink bits changed

One proposed fix is to make the test files in .hg/, but that will get us
in trouble if someone decides to use a symlink for .hg (not a terribly
unreasonable thing to do).

I propose to instead fix it by inserting these steps:

1a: MacHG immediately grabs a directory listing of repo/
4b: If there are pending events on repo/ upon return from calling
Mercurial,
4c: If it has the kFSEventStreamEventFlagMustScanSubDirs flag set, go to
step 2 - a genuine file change appeared during status
4d: Else, grab a directory listing of repo/ and compare it against the
one from 1a, if there are any changes, go to step 2

By confirming that the listings in 1a and 4d match, we can prove that
nothing has slipped by us in the affected directory while Mercurial was
doing its thing. This test should be pretty cheap as all the data will
generally be cache hot, moderately sized, and recursion isn't necessary.

Also note that it's possible to add:

1b: If we have a cached copy, compare it. If it's unchanged, we're done
- no need to call hg.

..which might be nice for dealing with temp files created by editors,
compilers, etc.

-- 
Mathematics is the supreme nostalgia of our time.