D3960: worker: use one pipe per posix worker and select() in parent process
Yuya Nishihara
yuya at tcha.org
Wed Jul 18 07:36:37 EDT 2018
> @@ -138,7 +138,15 @@
> oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler)
> ui.flush()
> parentpid = os.getpid()
> + pipes = []
> for pargs in partition(args, workers):
> + # Every worker gets its own pipe to send results on, so we don't have to
> + # implement atomic writes larger than PIPE_BUF. Each forked process has
> + # its own pipe's descriptors in the local variables, and the parent
> + # process has the full list of pipe descriptors (and it doesn't really
> + # care what order they're in).
> + rfd, wfd = os.pipe()
> + pipes.append((rfd, wfd))
> # make sure we use os._exit in all worker code paths. otherwise the
> # worker may do some clean-ups which could cause surprises like
> # deadlock. see sshpeer.cleanup for example.
> @@ -175,8 +183,10 @@
> finally:
> os._exit(ret & 255)
> pids.add(pid)
> - os.close(wfd)
> - fp = os.fdopen(rfd, r'rb', 0)
> + fps = []
> + for rfd, wfd in pipes:
> + os.close(wfd)
> + fps.append(os.fdopen(rfd, r'rb', 0))
This isn't enough. For child processes, all pipe fds except for the last
wfd have to be closed at the beginning of `workerfunc()`.
> + rlist, wlist, xlist = select.select(fps, [], fps)
Can you rewrite it to use the selectors module? commandserver.py has an
example.
More information about the Mercurial-devel
mailing list