[PATCH 1 of 6] socketserver: backport a less sucky socketserver from python 3.5.1

[Yuya Nishihara]

On Sat, 14 May 2016 14:50:28 +0100, Jun Wu wrote:
> On 05/14/2016 03:58 AM, Yuya Nishihara wrote:
> > Sorry for unclear words. I meant I didn't see a reason to make PreforkMixIn
> >  a mix-in of SocketServer. PreforkMixIn does its own job alone except for a
> >  trivial things, bind() and listen().  
> 
> I think {Forking,Threading}MixIn are doing similar things?

No, IMHO, they are designed to interact through the interface defined by
BaseServer, process_request().

If I were urged to implement a prefork server using SocketServer-like design,
I would add an isolated master server and make it create a SocketServer's
server after fork(). That's because the master does nothing about "serving"
in a sense of SocketServer.

  class StreamServer:
      # like TCPServer, but takes active socket object; assumes that we have it

  class PreforkMasterServer:
      def serve_forever(self):
          # spawn_workers_forever()

      def create_worker(self):
          pid = os.fork()
          if pid == 0:
              # no Forking nor Threading MixIn because this server handles
              # a single request at a time, which is bare SocketServer
              s = StreamServer(self.socket, self.RequestHandlerClass)
              s.serve_forever()
              os._exit()
          ...

> My understanding about mixins is they will use some other methods from
> the duck-typing class and add new features. I dislike a mixin overriding
> method. However since it's what existing code does, I'm okay with that.
> 
> The most famous mixin to me is Ruby's "Enumerable", it requires an "each"
> method and adds a lot of useful methods.

Enumerable is designed to extend the interface on top of #each, whereas
SocketServer's mixin is IMHO to provide a concrete implementation.

> > - and the problem is getting worse if there are more than one mix-ins that
> > override the same method. the order is controlled by how classes are
> > listed.  
> 
> True. But there is no such case yet and the behavior is defined, so I'm not
> too worried about this.

Well, we already did it, AutoExitMixIn.process_request.

> 3 classes and 4 mixins result in 12 reasonable combinations:
> 
>    (TCP, UDP, Unix) * (None, Threading, Forking, Preforking)
> 
> I don't think there is a better way to do code reuse.

Perhaps I would extract utility functions. No need to mess things up by
putting everything into one object.

> >> It's just a few lines to switch between them.  
> >
> > at the cost of making a mix-in class perfectly fit to SocketServer.  
> 
> I think I have made it fit.
> 
> See https://bitbucket.org/quark-zju/hg-draft/commits/6cfa29.

This time I can say serve_forever() is black magic, with no doubt.

I don't mean we shouldn't do that. Simply I don't understand why we have
to stick to the SocketServer in spite of its bugs on our mainline Python
version.

> > I care about the complexity of SocketServer utterly unnecessary for our
> > commandserver. There are two choices, and I prefer (b) because it's good
> > time to get rid of SocketServer.  
> 
> I really hate defending a Python stdlib.
> 
> I prefer shorter code, as long as it has reasonable good maintainability.
> I think you need a lot more lines to do the same thing.

Ok, please give me a few weeks to show how complicated it would be to write
a simple fork-per-request server without using the Python SocketServer. It
shouldn't be too late to choose your approach after that.

(I'm not fulltime and sadly I found 5 hours of sleep wasn't enough for me
these days. damn I'm getting old)

> And I guess other people from the community would prefer not maintaining a
> re-invented socketserver.

I'll never re-invent it, I regret choosing SocketServer for commandserver.