D3960: worker: use one pipe per posix worker and select() in parent process

Yuya Nishihara yuya at tcha.org
Wed Jul 18 07:36:37 EDT 2018


> @@ -138,7 +138,15 @@
>      oldchldhandler = signal.signal(signal.SIGCHLD, sigchldhandler)
>      ui.flush()
>      parentpid = os.getpid()
> +    pipes = []
>      for pargs in partition(args, workers):
> +        # Every worker gets its own pipe to send results on, so we don't have to
> +        # implement atomic writes larger than PIPE_BUF. Each forked process has
> +        # its own pipe's descriptors in the local variables, and the parent
> +        # process has the full list of pipe descriptors (and it doesn't really
> +        # care what order they're in).
> +        rfd, wfd = os.pipe()
> +        pipes.append((rfd, wfd))
>          # make sure we use os._exit in all worker code paths. otherwise the
>          # worker may do some clean-ups which could cause surprises like
>          # deadlock. see sshpeer.cleanup for example.
> @@ -175,8 +183,10 @@
>                  finally:
>                      os._exit(ret & 255)
>          pids.add(pid)
> -    os.close(wfd)
> -    fp = os.fdopen(rfd, r'rb', 0)
> +    fps = []
> +    for rfd, wfd in pipes:
> +        os.close(wfd)
> +        fps.append(os.fdopen(rfd, r'rb', 0))

This isn't enough. For child processes, all pipe fds except for the last
wfd have to be closed at the beginning of `workerfunc()`.

> +                rlist, wlist, xlist = select.select(fps, [], fps)

Can you rewrite it to use the selectors module? commandserver.py has an
example.


More information about the Mercurial-devel mailing list