[petsc-dev] Thoughts on pushing current CI infrastructure to the next level

[Jed Brown]

Karl Rupp <rupp at iue.tuwien.ac.at> writes:

> On 4/25/19 6:53 PM, Jed Brown wrote:
>> Karl Rupp via petsc-dev <petsc-dev at mcs.anl.gov> writes:
>> 
>>> With some effort we can certainly address 1.) and to some extent 3.),
>>> probably 4.) as well, but I don't know how to solve 2.) and 5.) with
>>> Jenkins. Given that a significant effort is required for 1.), 3.) and
>>> 4.) anyway, I'm starting to get more and more comfortable with the idea
>>> of rolling our own CI infrastructure (which has been suggested in some
>>> of Barry's snarky remarks already ;-) ). Small Python scripts for
>>> executing the tests and pushing results to Bitbucket as well as a
>>> central result storage can replicate our existing setup with a few lines
>>> of codes, while being much more flexible.
>> 
>> I think further commitment to Bitbucket would be a liability.
>
> Suggestions? Github, Gitlab, something else?

I like both of these significantly more than Bitbucket.

>> On existing open source CI tools, I think looking at how the project
>> itself uses CI is a good indicator.  Some examples of recent PRs with
>> test failures, see what needs to be done to narrow down what failed.
>> 
>> https://github.com/buildbot/buildbot/pull/4726
>> 
>> https://github.com/drone/drone/pull/2363
>> 
>> https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27652 (click Expand on the test failure)
>> 
>> https://github.com/jenkinsci/jenkins/pull/3991
>
> All of these offer more than what we have now. And yet, many offer 
> features we will never need and force upon us some constraints on our 
> workflow. Maybe other CI tools are better suited for PETSc than Jenkins 
> - I don't know, but maybe others know :-)
>
> Our requirements for a package-manager-like PETSc with all its possible 
> external packages and weird software stacks on supercomputers are 
> different to 'standard' software, unfortunately.

We could automate that better.  Any flexible CI/CD system can work for
us, provided it's easy to add agents.  I've personally set that up for
Jenkins and Gitlab-CI.  The latter is much more convenient for
subscribing agents.  I don't have personal experience with Buildbot and
Drone.