Making tests hashing-algorithm-agnostic

[Kyle Lippincott]

On Wed, Oct 29, 2014 at 8:54 AM, Mads Kiilerich <mads at kiilerich.com> wrote:

> On 10/28/2014 08:51 PM, Kyle Lippincott wrote:
>
>> I want to play around with different hashing algorithms, and the current
>> hard-coded hashes in the tests make that very difficult.
>>
>
>  Thoughts?
>>
>
> I do not see how swappable hash algorithms possibly could be a feature we
> should care about in the test suite.
>

Because otherwise you can't run the tests and get any useful information
from them.  It's the same reasoning behind b87acfda5268, where generaldelta
changed a bunch of 'base' values in 'hg debugindex' output, so the tests
were modified to make them agnostic to this change.


> For local testing while hacking, you can apply whatever hacks gets the job
> done for you.
>

There aren't really any local hacks available.  It'd take a week, at least,
to replicate the kind of stuff I'm doing here, which isn't pleasant and
conducive to experimentation.


>
> The actual hashes matters for some tests. Ignoring hash changes will
> probably make it very hard to debug and is not a good idea.
>

Right, that's why I want to have a way of keeping the hash values (the
final part of my original email, that basically just verifies that what was
captured matched what we expected to see).


>
> Instead, I can imagine putting some hacks in the code that for all content
> calculates both hashes and append them to a file in /tmp . Once that has
> been created, it will be quite trivial to make a search'n'replace, either
> in run-tests to replace your hashes with the good old ones in the test
> output, or just modifying the test files once and for all. The latter would
> probably also automatically handle the places where a test specify hashes
> explicitly ... except for the places where short prefixes are specified.
> You will still see some failures in places where the test depends on the
> sort order of hashes or uniqueness of short prefixes.


Putting that code in the actual hg binary would likely not fly, but getting
an extension to do it would be possible.  My biggest concern with this,
though, is what happens during the sha1 -> sha256 transition?  Do the tests
keep specifying only the sha1 hashes, or do we build a replacement map, or
just have two completely separate tests directories, or..?  Coming up with
an answer now that is as easy as possible for experimenting with even less
'likely' transitions would be a net win, I think.

(Also, only one test that I found actually depends on the sort order of
hashes, and there were no places where a short prefix collided in sha256
vs. sha1, mostly because 'short' prefix is generally 12 characters, and the
shortest I saw was 6, iirc)


Regardless, Matt has requested we not discuss this while the freeze is on,
so I'm going to mute the thread for a few days so that I'm not tempted to
respond. :/


>
> /Mads
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20141029/3492a521/attachment.html>