[PATCH] largefiles: makes verify to work on local content by default (issue4242) (BC)

[Piotr Listkiewicz]

>
>  It is ok that everybody is responsible for their own repositories. Remote
> repositories should be checked by running verify locally on the remote
> server.


I think that there is a general agreement that verify should work only on
local store (repo/hg/largefile + user cache) by default, this is a good
base for further discussions what options/parameters verify should have.

Largefiles to check:
> * referenced in specific revisions (currently only available as the
> default of checking current revision)
> * referenced in repo (--lfa)
> * all largefiles in store (cd .hg/largefiles && sha1sum -c <(ls
> ?????????*| while read f; do echo $f $f; done))
> Locations to check:
> * check repo store
> * check "user cache"
> * check default remote server (currently apparently inevitable)
> * first of these (should probably be default)
> Kind of check:
> * check existence (default)
> * check actual hash (--lfc but not possible remote without downloading)


I agree, it should be discussed what options should be included in check

It seems to me like there currently are two bugs (in design or
> implementation):
> 1. 'hg verify --large' currently invokes remote statlfile for all files,
> even if they already are available locally.
> 2. 'hg verify --large' currently sends one remote statlfile command once
> for each file. It should use batching.


If we agree that verify works on local contents by default, problem with
batching will exist only with --remote option.

2016-04-12 23:07 GMT+02:00 Mads Kiilerich <mads at kiilerich.com>:

> On 03/02/2016 02:05 PM, liscju wrote:
>
>> # HG changeset patch
>> # User liscju <piotr.listkiewicz at gmail.com>
>> # Date 1456917749 -3600
>> #      Wed Mar 02 12:22:29 2016 +0100
>> # Node ID 184b0386fad4aff1ec64f0076c74c13e2cf5d036
>> # Parent  c7f89ad87baef87f00c507545dfd4cc824bc3131
>> largefiles: makes verify to work on local content by default (issue4242)
>> (BC)
>>
>
> Sorry for late response. Here comes something.
>
>
> Conceptually, there can be many different variations of verification of
> largefile repos:
>
> Largefiles to check:
> * referenced in specific revisions (currently only available as the
> default of checking current revision)
> * referenced in repo (--lfa)
> * all largefiles in store (cd .hg/largefiles && sha1sum -c <(ls
> ?????????*| while read f; do echo $f $f; done))
>
> Locations to check:
> * check repo store
> * check "user cache"
> * check default remote server (currently apparently inevitable)
> * first of these (should probably be default)
>
> Kind of check:
> * check existence (default)
> * check actual hash (--lfc but not possible remote without downloading)
>
> I don't know which combinations we want to support. The current options
> might be a bit arbitrary. If we change/add new options, we should make sure
> they are less arbitrary or represent common use cases.
>
> I don't think we should support remote (wireproto) hash checking. That
> would make it too easy to make a cpu load DoS. It is more managable if
> "attackers" have to use bandwidth to hash remote largefiles. It is ok that
> everybody is responsible for their own repositories. Remote repositories
> should be checked by running verify locally on the remote server.
>
>
> For the patch, I would prefer to have a first "add test coverage" /
> "demonstrate problem" patch - that would make it easier to spot what the
> problem is and discuss the problem and fixes.
>
>
> It seems to me like there currently are two bugs (in design or
> implementation):
>
> 1. 'hg verify --large' currently invokes remote statlfile for all files,
> even if they already are available locally.
>
> 2. 'hg verify --large' currently sends one remote statlfile command once
> for each file. It should use batching.
>
> If these two were fixed, I think it would be less of a problem (or perhaps
> no problem) that it falls back to using remote. It would exactly mirror
> what happens on actual use.
>
> It seems like it currently handles lack of remote server availability by
> simple reporting the largefiles it needs missing. I think that is fine.
>
>
> /Mads
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20160413/f9e5e9ed/attachment.html>