[petsc-dev] PETSc goes Jenkins

[Karl Rupp]

Hi Patrick,

> Once tuning is complete, how is one intended to interpret the nice green 
> check marks? "The library compiles" or "All the tests passed"?

So far the green checkmark essentially means "The library compiles". We 
are working towards "All the tests passed", but that requires some more 
tinkering in processing test output. That is, we need to extract 
information about failed tests and present that in a prominent way.

Best regards,
Karli


> 
> I ask because in the demo PR there is the reassuring check mark and "3 
> of 3 builds passed", even though failed tests are reported (timeouts).
> 
> 
> 2018-07-20 3:35 GMT+02:00 Karl Rupp <rupp at iue.tuwien.ac.at 
> <mailto:rupp at iue.tuwien.ac.at>>:
> 
>     Hi all,
> 
>     we now have a first step towards full continuous integration via
>     Jenkins completed. Thus, every new pull request that is (re-)based
>     on a commit in master not older than today will be automatically
>     tested with a subset of common tests that are intended to expose the
>     most frequent issues. This, in particular, includes configurations
>     with 64 bit integers as well as complex arithmetic.
> 
>     The integration of Jenkins into Bitbucket is smooth: You will notice
>     on our demo pull request
> 
>     https://bitbucket.org/petsc/petsc/pull-requests/1039/jenkinsfile-for-build-pipelines-tied-to/diff
>     <https://bitbucket.org/petsc/petsc/pull-requests/1039/jenkinsfile-for-build-pipelines-tied-to/diff>
>     that on the right it says "3 of 3 builds passed". If you click on
>     the link, you will get further details on the individual builds and
>     find further links to the test output stored on the Jenkins server.
> 
>     Implications on our development workflow: Currently 'next' gets
>     (ab)used for all kinds of portability tests. As a consequence, every
>     buggy merge clogs the whole integration pipeline, making it hard to
>     integrate other PRs. With the Jenkins server in place, all pull
>     requests will receive a good share of portability testing *before*
>     they reach next. This reduces the burden on next, (hopefully)
>     leading to faster code integration.
> 
>     Corollary: I strongly encourage all PETSc developers to use issue
>     pull requests rather than merging to next directly (use your own
>     judgment for exceptions!).
> 
>     Please note that we are still fine-tuning various aspects of the
>     Jenkins infrastructure (location of the Jenkins server, which test
>     nodes to use, which configurations to test, etc.). Most of these
>     things are changes under the hood, though. If something still
>     bubbles up and causes the testing to choke, please be considerate
>     with us ;-)
> 
>     Finally, I'd like to explicitly thank Alp Dener for his help on
>     getting Jenkins to run smoothly. Any credit should go to him.
> 
>     Best regards,
>     Karli
> 
>