Some thoughts on possible improvements to Revlog.
Things that could use fixing:
- no uncompressed revision size stored
- SHA1 hash is potentially too weak
- compression context for deltas is often too small
- offset range is limited to 4MB
- some metadata is indicated by escaping in the data
The current index layout is:
- 4 bytes: offset
- 4 bytes: compressed length
- 4 bytes: base revision
- 4 bytes: link revision
- 20 bytes: nodeid
- 20 bytes: parent 1 nodeid
- 20 bytes: parent 2 nodeid
72 bytes total
Possible future layout:
- 2 bytes: flags
- 6 bytes: offset (allows for 256TB of compressed history per file)
- 4 bytes: compressed length
- 4 bytes: uncompressed length
- 4 bytes: base revision
- 4 bytes: link revision
- 4 bytes: parent 1 revision
- 4 bytes: parent 2 revision
- 32 bytes: nodeid
64 bytes total
Also, it may be interesting to compress entire spans with a single gzip context, with flushes between revisions. This will possibly result in much better compression.