[PATCH STABLE] worker: wait worker pid explicitly

Yuya Nishihara yuya at tcha.org
Mon Jul 25 10:42:34 EDT 2016


On Mon, 25 Jul 2016 14:45:41 +0100, Jun Wu wrote:
> Excerpts from Yuya Nishihara's message of 2016-07-25 21:49:35 +0900:
> > On Mon, 25 Jul 2016 10:27:17 +0100, Jun Wu wrote:
> > > Excerpts from Yuya Nishihara's message of 2016-07-24 15:32:10 +0900:
> > > > Also it doesn't work under vanilla commandserver which replaces ui.fout by
> > > > a pseudo file object.
> > > 
> > > I think commandserver needs some lock mechanism to protect its I/O. flock
> > > is a good choice wherever supported. The problem is not only limited to
> > > workers but also any other places using threads / processes.
> > 
> > No idea how flock() will work for duplicated fds. Are we going to make a
> > temporary lock file?
> 
> TIL flock and fcntl(F_SETLKW) are not the same. The latter works in this
> case without an extra file.

That seems getting more complicated and platform-dependent. But thanks for the
tip, I had no knowledge about F_GET/SETLK.

> > As long as underlying fwrite() is thread-safe, the issue can be avoided
> > by writing header + data by single write() call. That can't be true for
> > fork(), but IIRC worker.py is the only place we have to care.
> 
> Ideally, workers are threads. But I understand we use processes to
> workaround Python's GIL. It feels like reinventing stdio because of the GIL.
>
> The main thing I dislike about the pipe plan is unnecessary memory copy /
> CPU usage. If the files being updated are many, their paths can be a lot of
> bytes.

IMHO that's okay because they explicitly request a bunch of outputs by
--verbose. (I haven't measured how slow that would be, so I might be wrong.)

> Assuming status lines are less than 4KB, make sure write uses a string
> ending with "\n" (aka. line buffered, stdbuf -oL) should solve this issue
> without any locks.

and line-buffered output would increase the cost of write() syscalls anyway.


More information about the Mercurial-devel mailing list