[PATCH 3 of 4 V2] chgcache: implement the background preloading thread

[Yuya Nishihara]

On Wed, 1 Mar 2017 08:38:03 -0800, Jun Wu wrote:
> Excerpts from Yuya Nishihara's message of 2017-03-01 21:56:08 +0900:
> > On Wed, 22 Feb 2017 18:16:10 -0800, Jun Wu wrote:
> > > # HG changeset patch
> > > # User Jun Wu <quark at fb.com>
> > > # Date 1487813979 28800
> > > #      Wed Feb 22 17:39:39 2017 -0800
> > > # Node ID 28571825744fb4f4b424385f55afa9484532ef43
> > > # Parent  630457e88fa0485bce7822345ea640b5cdcb9b8e
> > > # Available At https://bitbucket.org/quark-zju/hg-draft 
> > > #              hg pull https://bitbucket.org/quark-zju/hg-draft  -r 28571825744f
> > > chgcache: implement the background preloading thread
> > > 
> > > The background preloading thread is meant to used by chg master, and it sets
> > > up the IPC channel.
> > 
> > > +    def _preloadloop(self, interval=0.5):
> > > +        while not self._shouldexit.is_set():
> > > +            try:
> > > +                atimestr, path = self._ipc.recv().split(' ', 1)
> > > +                atime = float(atimestr)
> > > +            except Exception: # ignore errors
> > > +                pass
> > > +            else:
> > > +                if path in _repocaches and _repocaches[path][0] >= atime:
> > > +                    # this repocache is up-to-date, no need to update
> > > +                    continue
> > > +                now = time.time()
> > > +                # TODO update repocache properly
> > 
> > Another concern just came up. Do you know what will happen if the process
> > is fork()ed while the preloader thread is actively loading cache?
> > 
> > Only the main thread (which called fork()) would be copied, so the preloader
> > would stop. That should be okay, but the other resources such as files and
> > mutexes (if any) would be copied.
> 
> Right. It's possible to leak some fds, where I think it's probably fine. I
> guess Python will deal with mutexes created by Python code.

I hope so, but googling "fork thread" or "fork thread python" made me feel
bad. I haven't read them carefully, but they seemed to say "don't do it."