[PATCH 1 of 5] transaction: change the on disk format for backupentries

Pierre-Yves David pierre-yves.david at ens-lyon.org
Fri Nov 14 13:09:11 CST 2014



On 11/14/2014 07:01 PM, Gregory Szorc wrote:
> On 11/14/14 12:30 AM, Pierre-Yves David wrote:
>>
>>
>> On 11/14/2014 06:08 AM, Gregory Szorc wrote:
>>> On 11/13/14 3:39 PM, Pierre-Yves David wrote:
>>>> # HG changeset patch
>>>> # User Pierre-Yves David <pierre-yves.david at fb.com>
>>>> # Date 1415152366 0
>>>> #      Wed Nov 05 01:52:46 2014 +0000
>>>> # Node ID dba745ba04b9bc614d422e2e88b362a3bbda6948
>>>> # Parent  e44399c494ab23cefb8c99087529f9b4b06d398e
>>>> transaction: change the on disk format for backupentries
>>>
>>> So I guess we aren't going with the solution that was brainstormed at
>>> the Munich sprint? (See "Transactions" in
>>> https://titanpad.com/mercurial32-sprint)
>>>
>>> Or are you going with an easy solution to transactions before the
>>> complicated one (that will support reader locks)?
>>
>> I'm starting for a simple solution first. The complexe solution
>> discussed at the munich sprint is "useless" if nobody is playing the
>> transaction game. So I first need to consolidate our transaction logic
>> to ensure basic property:
>>
>> - All contents written during the transaction is only made visible at
>> the end of the transaction
>> - All content written during the transaction is made available to hooks
>> and hooks only right before we commit.
>> - Any temporary file are properly cleaned up
>> - Any file involved in properly backed up in case of error.
>>
>> Once I've this true, I can move to the next step: Change the format to
>> ensure an atomic "write" of the data from a reader perspective. But I
>> needs some to write atomically in the first place ;-)
>
> Makes sense to me.
>
> I am very excited about seeing these patches. As you may know, cache
> invalidation causing an expensive rebuild is a big problem for large
> repositories. We are seeing these expensive rebuilds occur after a
> pretxnchangegroup hook fails a push, for example. I can't wait to deploy
> 3.3 at Mozilla!
>
> Of course, we still see cache invalidation after strip operations. We'll
> need memctx everywhere, hidden changesets enabled by default, and people
> to stop using strip-based workflows (mq) before this is fully resolved,
> I fear. But this transaction work is refreshing nonetheless.

Note: my current plan it to put the infrastructure in place (using the 
simplest cache we have to test it: hidden cache). I then expect an army 
of mozillian leprechaun to pop and do the work to move bigger cache in 
the transaction.

-- 
Pierre-Yves David


More information about the Mercurial-devel mailing list