[PATCH 1 of 5 V4] testing: generate tests operations using Hypothesis

[David MacIver]

On 24 February 2016 at 21:16, Martin von Zweigbergk <martinvonz at google.com>
wrote:
>
> When running this test case, I get the following error (not surprisingly).
>
> +Traceback (most recent call last):
> +  File "~/hg/tests/test-verify-repo-operations.py", line 18, in <module>
> +    from hypothesis.extra.datetime import datetimes
> +ImportError: No module named hypothesis.extra.datetime
>
> I guess we should skip the test if Hypothesis is not installed, just
> like we do with e.g. test-convert-bzr-*.t.
>
>
OK, I'll add something to do that.


> Also, for people like me who don't know much about the Python
> ecosystem, how do I even install Hypothesis? Do I follow the
> instructions on
> http://python-packaging-user-guide.readthedocs.org/en/latest/installing/?
>

Yes. Just run "pip install hypothesis"


> Even then, I suppose some kind of package name or URL could be added
> to the test case?
>

OK.


>
> > @@ -0,0 +1,381 @@
> > +from __future__ import print_function, absolute_import
> > +
> > +"""Fuzz testing for operations against a Mercurial repository
> > +
> > +This uses Hypothesis's stateful testing to generate random repository
> > +operations and test Mercurial using them, both to see if there are any
> > +unexpected errors and to compare different versions of it."""
> > +
> > +import base64
> > +from contextlib import contextmanager
> > +import errno
> > +import os
> > +import pipes
> > +import shutil
> > +import silenttestrunner
> > +import subprocess
> > +
> > +from hypothesis.extra.datetime import datetimes
> > +from hypothesis.errors import HypothesisException
> > +from hypothesis.stateful import rule, RuleBasedStateMachine, Bundle
> > +import hypothesis.strategies as st
>
> nit: Does "from hypothesis import strategies as st" do the same thing?
> It would be more consistent with the surrounding lines.
>

For no particularly good reason my convention is that modules/packages are
imported qualified as 'import my.module as mm' and things from modules are
imported with 'from my import thing'. I'm happy to change it if you prefer
though.


> > +    # Section: Set up basic data
> > +    # This section has no side effects but generates data that we will
> want
> > +    # to use later.
> > +    @rule(
> > +        target=paths,
> > +        source=st.lists(files, min_size=1).map(
> > +            lambda l: os.path.join(*l)).filter(
> > +                lambda x: x and x[0] != "/" and x[-1] != "/"))
>
> Why is the call to filter() needed? Does os.path.join() on some
> platform return add an initial or final os.path.join() or is this just
> leftovers from an earlier version?


Hm. I'm actually not sure. I was going to say it was because files might be
the empty string due to the stripping of leading and trailing characters,
but that doesn't produce a leading slash on join.


> (Maybe that's also why
> "pathcharacters" is not called "filecharacters".)
>

Uh, good point. I'll change that.

>
> > +    def genpath(self, source):
> > +        return source
>
> It seems simpler to drop the "source" parameter and create it in the
> method instead. What's the reason for the current style? Some
> requirement from Hypothesis?
>
>
This is a requirement from Hypothesis because of how the rules are
constructed and used. It's ended up a bit awkward here. I'm trying to
figure out a better API to improve upon this.


>
> When does this happen (because mkdirp() seems to check whether the
> path already exists)?
>

When a parent of the path exists and is not a directory. So if we e.g. try
to create foo/bar/ but foo already exists and is an ordinary file.


> > +
> > +    @rule(s=st.none() | st.integers(0, 100))
>
> Is the '|' operator a synonym for st.one_of() used above? Or perhaps
> this form gives "none" a 1/102 probability and one_of() gives it a 1/2
> probability?
>
>
The former. I tend not to use one_of for pairs of strategies.

Thanks for review. Will send a new patch soon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20160225/6fa778d6/attachment.html>