D3960: worker: use one pipe per posix worker and select() in parent process

[yuja (Yuya Nishihara)]

yuja added a comment.


  > @@ -138,7 +138,15 @@
  > 
  >   oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler)
  >   ui.flush()
  >   parentpid = os.getpid()
  > 
  > +    pipes = []
  > 
  >   for pargs in partition(args, workers):
  > 
  > +        # Every worker gets its own pipe to send results on, so we don't have to
  >  +        # implement atomic writes larger than PIPE_BUF. Each forked process has
  >  +        # its own pipe's descriptors in the local variables, and the parent
  >  +        # process has the full list of pipe descriptors (and it doesn't really
  >  +        # care what order they're in).
  >  +        rfd, wfd = os.pipe()
  >  +        pipes.append((rfd, wfd))
  > 
  >   1. make sure we use os._exit in all worker code paths. otherwise the
  >   2. worker may do some clean-ups which could cause surprises like
  >   3. deadlock. see sshpeer.cleanup for example. @@ -175,8 +183,10 @@ finally: os._exit(ret & 255) pids.add(pid)
  > - os.close(wfd)
  > - fp = os.fdopen(rfd, r'rb', 0) +    fps = [] +    for rfd, wfd in pipes: +        os.close(wfd) +        fps.append(os.fdopen(rfd, r'rb', 0))
  
  This isn't enough. For child processes, all pipe fds except for the last
  wfd have to be closed at the beginning of `workerfunc()`.
  
  > +                rlist, wlist, xlist = select.select(fps, [], fps)
  
  Can you rewrite it to use the selectors module? commandserver.py has an
  example.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D3960

To: hooper, #hg-reviewers
Cc: yuja, mercurial-devel