Command Server

A server that allows communication with Mercurial's API over a pipe.

1. Rationale

Mercurial presents several barriers for third-party applications wishing to automate interaction:

The usual answer to these problems is to use its command line API which is:

The two primary downsides of this approach are:

Thus, the goal of the command server is to facilitate the creation of wrapper libraries that are:

2. Protocol

All communication with the server is done on stdin/stdout. The byte order used by the server is big-endian.

Data sent from the server is channel based, meaning a (channel, length) pair is sent before the actual data. The channel is a single character, while the length is an unsigned int (4 bytes). In the examples below, the length field is in plain text.

o
1234
<data: 1234 bytes>

that is 1234 bytes sent on channel 'o', with 1234 bytes of data following.

When starting the server, it will send a hello message on the 'o' channel. The message is sent as one chunk. It is composed of a \n separated list where each item is of the format:

<field name>: <field data>

<field name> is limited to [a-z0-9]+, and <field data> is field specific (cannot contain new lines).

Known fields are:

capabilities: capability1 capability2 ... capabilityN\n
encoding: UTF-8

At the most basic level, the server will support the 'runcommand' capability.

Clients should ignore unknown fields in the hello message, in case a new version of the server decides to update it with some important information.

More on channels below.

2.1. Encoding

Strings are encoded by default in Mercurial's local encoding. At the moment the encoding cannot be changed after server startup. To set it at startup, use HGENCODING. To query the server's encoding, see the 'getencoding' command.

Clients wanting to use Unicode should specify a UTF-8 encoding, but be aware that some responses will mix UTF-8 metadata and raw file contents. See EncodingStrategy for more information.

2.2. Channels

Channels are divided into two, required and optional. Required channels identifiers are uppercase. They cannot be ignored. If a client encounters an unexpected required channel, it should abort.

Optional channels identifiers are lowercase, and their data can be ignored.

Optional:

Required:

Input should be sent on stdin in the following format:

length
data

length = 0 sent by the client is interpreted as EOF by the server. The server will not ask for more than 4kb per request as to not fill up the pipe.

2.3. Commands

The server is running on an endless loop (until stdin is closed) waiting for commands. A command request looks like this:

commandname\n
<command specific request>

The server aborts upon unknown commands. Clients are expected to check what commands are supported by the server by consulting the capabilities.

2.3.1. runcommand

Run the command specified by a list of \0-terminated strings. An unsigned int indicating the length of the arguments should be sent before the list. Example:

runcommand\n
8
log\0
-l\0
5

Which corresponds to running 'hg log -l 5'.

The server responds with input/output generated by Mercurial on the matching channels. When the command returns, the server writes the return code (signed integer) of the command to the 'r'esult channel.

2.3.2. getencoding

Returns the servers encoding on the result channel.

client:

getencoding\n

server responds with:

r
5
ascii

2.4. Examples

2.4.1. runcommand

Complete example of a client running 'hg summary', right after starting the server:

(text in the server column is <channel>: <length>, where length is really 4 byte unsigned ints, not plain text like below)

server

client

notes

connected, waiting for hello message

o: 52
capabilities: runcommand getencoding\n
encoding: UTF-8

server is waiting for a command

runcommand\n
7
summary

client talks to server on stdin

starts running command

o: 27
parent: 14571:17c0cb1045e5

o: 3
tip

o: 1
\n

o: 53
paper, coal: display diffstat on the changeset page\n

o: 16
branch: default\n

o: 16
commit: (clean)\n

o: 18
update: (current)\n

r: 4
0

server finished running command, writes ret on the 'r' channel to the client

closes server stdin

client disconnects

server exits

client waits for server to exit

And another one with activity on the input channels too by running 'import -':

(starting after client read the hello message)

server

client

notes

server is waiting for a command

getencoding\n

r: 5
UTF-8

server responds with the encoding, then waits for the next command

runcommand\n
8
import\0
-

starts running command

o: 26
applying patch from stdin\n

l: 4096

server tells client to send it a line

21
# HG changeset patch\n

client responds with <length><line>

l: 4096

server processes line, asks for another one

...this goes on until the client has no more input

l: 4096

0

it responds with length=0

r: 4
0

server finished running command, writes ret on the 'r' channel to the client

closes server stdin

client disconnects

server exits

client waits for server to exit

3. Known issues

4. Example client

This is a minimal Python example to illustrate how to establish a connection and execute a command.

   1 import sys, struct, subprocess
   2 
   3 # connect to the server
   4 server = subprocess.Popen(['hg', 'serve', '--cmdserver', 'pipe'],
   5                           stdin=subprocess.PIPE, stdout=subprocess.PIPE)
   6 
   7 def readchannel(server):
   8     channel, length = struct.unpack('>cI', server.stdout.read(5))
   9     if channel in 'IL': # input
  10         return channel, length
  11     return channel, server.stdout.read(length)
  12 
  13 def writeblock(data):
  14     server.stdin.write(struct.pack('>I', len(data)))
  15     server.stdin.write(data)
  16     server.stdin.flush()
  17 
  18 # read the hello block
  19 hello = readchannel(server)
  20 print "hello block:", repr(hello)
  21 
  22 # write the command
  23 server.stdin.write('runcommand\n')
  24 writeblock('\0'.join(sys.argv[1:]))
  25 
  26 # receive the response
  27 while True:
  28     channel, val = readchannel(server)
  29     if channel == 'o':
  30         print "output:", repr(val)
  31     elif channel == 'e':
  32         print "error:", repr(val)
  33     elif channel == 'r':
  34         print "exit code:", struct.unpack(">l", val)[0]
  35         break
  36     elif channel == 'L':
  37         print "(line read request)"
  38         writeblock(sys.stdin.readline(val))
  39     elif channel == 'I':
  40         print "(block read request)"
  41         writeblock(sys.stdin.read(val))
  42     else:
  43         print "unexpected channel:", channel, val
  44         if channel.isupper(): # required?
  45             break
  46 
  47 # shut down the server
  48 server.stdin.close()

5. Libraries

A list of client libraries using the command server (feel free to add yours here):


CategoryDeveloper