[petsc-dev] Thoughts on pushing current CI infrastructure to the next level

[Jed Brown]

Karl Rupp via petsc-dev <petsc-dev at mcs.anl.gov> writes:

> With some effort we can certainly address 1.) and to some extent 3.), 
> probably 4.) as well, but I don't know how to solve 2.) and 5.) with 
> Jenkins. Given that a significant effort is required for 1.), 3.) and 
> 4.) anyway, I'm starting to get more and more comfortable with the idea 
> of rolling our own CI infrastructure (which has been suggested in some 
> of Barry's snarky remarks already ;-) ). Small Python scripts for 
> executing the tests and pushing results to Bitbucket as well as a 
> central result storage can replicate our existing setup with a few lines 
> of codes, while being much more flexible.

I think further commitment to Bitbucket would be a liability.

On existing open source CI tools, I think looking at how the project
itself uses CI is a good indicator.  Some examples of recent PRs with
test failures, see what needs to be done to narrow down what failed.

https://github.com/buildbot/buildbot/pull/4726

https://github.com/drone/drone/pull/2363

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27652 (click Expand on the test failure)

https://github.com/jenkinsci/jenkins/pull/3991