Note:

This page is primarily intended for developers of Mercurial.

Multicore Plan

Adding multicore support for Mercurial

1. Introduction

Various Mercurial operations could be faster if they were able to run on multiple cores. These include:

Given that CPU cores are no longer getting significantly faster, using multiple cores allows using the available processing power of a workstation more efficiently.

2. Strategy

Unfortunately, there is no standard paradigm that lets us easily and efficiently use multiple cores.

So our strategy will be to use a hybrid approach: fork() on Unix and Python threads on Windows. This hybrid will be called a "worker" and will be subject to the constraints of both models:

Workers are managed by a generic dispatcher function that takes a work function and a list of jobs. The dispatcher creates a worker pool, farms out jobs to available workers, collects and combines results, and shuts down the pool on completion.

Code that wants to support multicore should implement a single path (ie worker function) using the dispatcher for both single-threaded and multicore use.


CategoryDeveloper CategoryNewFeatures

MulticorePlan (last edited 2012-05-21 07:08:03 by ThomasArendsenHein)