[petsc-dev] Nightly tests quick summary page

[Matthew Knepley]

On Wed, Jan 23, 2013 at 4:37 PM, Karl Rupp <rupp at mcs.anl.gov> wrote:

> Dear PETScians,
>
> I've adopted the HTML summary page script for the style checks a little in
> order to have a quick overview of the results of our nightly tests:
>   http://krupp.iue.tuwien.ac.at/**petsc-test/<http://krupp.iue.tuwien.ac.at/petsc-test/>
> It doesn't do anything fancy, particularly no automatic bisection or the
> like, but it gives an overview of the Nightly results in seconds (which are
> not overly good at the moment...).
>

Yep, this is nice.


> Assuming that there is no question about the need for improving our
> current automatic testing environment, it's merely a question of how far we
> want to go. Regarding overall productivity, I made good experiences with
> just using CTest and setting the summary page as the start page of my
> browser. It's not overly complicated an can be integrated with reasonable
> effort into our CMake build system.
>
> We could also do all the fancy things provided by continuous integration
> systems. Sadly, I don't have any experience on that front. Regardless, I
> still prefer rather simple systems which are in daily use and fairly robust
> rather than a complex, it-can-do-anything-for-you-if-**you-find-the-right-switch
> system that is never set up...
>
> Any thoughts/experiences? I know Jed has good plans on a testing
> environment, so this is not meant to be an assault on him ;-)
>

I am always skeptical of big programs to overall a large piece of
infrastructure that works fairly well.

However, there is a really simple thing that we need which would make us
much much much better
at using our own tests. We need a better way to test numerical output. I am
not sure what the right
thing to do is, or I would have already done it.

The current best solution is to print fewer digits, which judging from the
HTML page is not sufficient.
I think that current PETSc output is so stylized that parsing output is
feasible, and would allow nice
diffs with tolerances tailored to the type of output and individual test.

    Matt


> Best regards,
> Karli
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130123/87630e25/attachment.html>