[PATCH V6] run-tests: added '--json' functionality to store test result in json file

Pierre-Yves David pierre-yves.david at ens-lyon.org
Thu Sep 4 07:09:57 CDT 2014



On 09/03/2014 12:08 AM, Anurag Goel wrote:
> # HG changeset patch
> # User anuraggoel <anurag.dsps at gmail.com>
> # Date 1409695563 -19800
> #      Wed Sep 03 03:36:03 2014 +0530
> # Node ID 92789b9327d0af71964ccbcff3f34caa17f5b7a7
> # Parent  188b8aa2120b03eead618ba150319074f4e3b42b
> run-tests: added '--json' functionality to store test result in json file
>
> This patch added a new functionality '--json'. While testing, if '--json'
> is enabled then test result data gets stored in newly created "report.json"
> file in the following format.
>
> testreport ={
>      "test-success.t": {
>          "result": "success",
>          "time": "2.041"
>      }
>      "test-failure.t": {
>          "result": "failure",
>          "time": "4.430"
>      }
>      "test-skip.t": {
>          "result": "skip"
>          "time": "3.754"
>      }
> }
>
> This "report.json" file will further accessed by html/javascript file for
> graph usage.
>
> diff -r 188b8aa2120b -r 92789b9327d0 tests/run-tests.py
> --- a/tests/run-tests.py	Sat Aug 30 18:44:59 2014 +0200
> +++ b/tests/run-tests.py	Wed Sep 03 03:36:03 2014 +0530
> @@ -184,6 +184,8 @@
>       parser.add_option("-t", "--timeout", type="int",
>           help="kill errant tests after TIMEOUT seconds"
>                " (default: $%s or %d)" % defaults['timeout'])
> +    parser.add_option("--json", action="store_true",
> +        help="store test result data in 'report.json' file")
>       parser.add_option("--time", action="store_true",
>           help="time how long each test takes")
>       parser.add_option("--tmpdir", type="string",
> @@ -1419,6 +1421,42 @@
>               finally:
>                   xuf.close()
>
> +        if self._runner.options.json:
> +            try:
> +                if sys.version_info < (2, 7):
> +                    import simplejson as json
> +                else:
> +                    import json

This should go at be begining of the file, with the except clause 
setting `json = None`. Then you can tests here if `json is None` and 
bail out appropriately. This would prevent such wide try except

> +
> +                fp = open(os.path.join(self._runner._testdir,
> +                          'report.json'), 'w')

'report.json' should be aligned with the second parents (since it is 
enclosed in that second parents.


> +                try:
> +                    timest = []
> +                    for test, cuser, csys, real in result.times:
> +                        timest.append((test, real))
> +                    timesd = dict(timest)

Why don't you use a temporary list? you could build the dictionnary 
directly.

> +
> +                    outcome = []

Same here Outcome seems to be never read as a list. why not making a 
dictionary in the first place?

> +                    for tc in result.successes:
> +                        testresult = {'result': 'success',
> +                                 'time': ('%0.3f' % timesd[tc.name])}

You have wrong alignment here too

> +                        outcome.append((tc.name, testresult))
> +                    for tc, err in sorted(result.faildata.iteritems()):
> +                        testresult = {'result': 'failure',
> +                                 'time': ('%0.3f' % timesd[tc])}

You have wrong alignment here too

> +                        outcome.append((tc, testresult))
> +                    for tc, reason in result.skipped:
> +                        testresult = {'result': 'skip',
> +                                 'time': ('%0.3f' % timesd[tc.name])}

You have wrong alignment here too

> +                        outcome.append((tc.name, testresult))
> +                    testdata = dict(outcome)
> +                    fp.writelines(("testreport =", (json.dumps(testdata,
> +                                    sort_keys=True, indent=4))))
> +                finally:
> +                    fp.close()
> +            except ImportError:
> +                pass

If --json was specified without a --json module. we should crash. not 
silently ignores it.

> +
>           self._runner._checkhglib('Tested')
>
>           self.stream.writeln('# Ran %d tests, %d skipped, %d warned, %d failed.'
> diff -r 188b8aa2120b -r 92789b9327d0 tests/test-run-tests.t
> --- a/tests/test-run-tests.t	Sat Aug 30 18:44:59 2014 +0200
> +++ b/tests/test-run-tests.t	Wed Sep 03 03:36:03 2014 +0530
> @@ -357,3 +357,40 @@
>     Skipped test-failure.t: blacklisted
>     # Ran 0 tests, 2 skipped, 0 warned, 0 failed.
>
> +test for --json
> +==================
> +
> +  $ $TESTDIR/run-tests.py --with-hg=`which hg` --json
> +
> +  --- $TESTTMP/test-failure.t
> +  +++ $TESTTMP/test-failure.t.err
> +  @@ -1,4 +1,4 @@
> +     $ echo babar
> +  -  rataxes
> +  +  babar
> +   This is a noop statement so that
> +   this test is still more bytes than success.
> +
> +  ERROR: test-failure.t output changed
> +  !.s
> +  Skipped test-skip.t: skipped
> +  Failed test-failure.t: output changed
> +  # Ran 2 tests, 1 skipped, 0 warned, 1 failed.
> +  python hash seed: * (glob)
> +  [1]
> +
> +  $ cat report.json
> +  testreport ={
> +      "test-failure.t": [\{] (re)
> +          "result": "failure",
> +          "time": "\s*[\d\.]{5}" (re)
> +      },
> +      "test-skip.t": {
> +          "result": "skip",
> +          "time": "\s*[\d\.]{5}" (re)
> +      },
> +      "test-success.t": [\{] (re)
> +          "result": "success",
> +          "time": "\s*[\d\.]{5}" (re)
> +      }
> +  } (no-eol)
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
>

-- 
Pierre-Yves David


More information about the Mercurial-devel mailing list