[petsc-dev] broken nightlybuilds (next vs next-tmp)

[Matthew Knepley]

On Sat, Nov 11, 2017 at 12:53 PM, Satish Balay <balay at mcs.anl.gov> wrote:

> On Sat, 11 Nov 2017, Matthew Knepley wrote:
>
> > On Sat, Nov 11, 2017 at 12:37 PM, Satish Balay <balay at mcs.anl.gov>
> wrote:
> >
> > > On Sat, 11 Nov 2017, Matthew Knepley wrote:
> > >
> > > > > In the long term - Barry wants to get rid of next..
> > > >
> > > >
> > > > 1) I think next really prevents master from getting screwed up
> (witness
> > > > next)
> > > >
> > > > 2) I think we are actually finding interaction bugs there.
> > > >
> > > > Are those points wrong, or is there another way to do these things?
> > >
> > > Next is an intergration testing mechanism. The prerequisite for it [I
> > > think] is - one should test the branch properly before merging to
> > > next. However we are not doing proper testing before merge to next -
> > > and relying on next to do this part aswell.
> > >
> > > So with current next - it one has to figure out which branches are
> > > breaking the tests [takes time - which most of us are not doing] - and
> > > then hope it gets fixed quickly. Otherwise next stay broken for a long
> > > time [and other branches in next - which could be clean - don't
> > > receive sufficient confidence to graduate to master]
> > >
> > > So Barry's thought wrt getting next is to have a better way of testing
> > > feature branch one wants to test (i.e master+feature). Its not clear
> > > to me how many integration issues we've addressed with our current
> > > next model. [Its mostly been indvidual branch issues]
> > >
> > > Also if feature-1 and feature-2 are feature branches that are tested
> > > in next [wrt integration]. The following should be equivalent to
> > > testing 'master + feature1 + feature2' - aka current next model:
> > >
> > > 1. test master+feature1
> > > 2. success => merge feature1 to master
> > > 3. tets master+ feature2
> > > 3. success => merge feature2 to master
> > >
> > > Note: my next-tmp is an attempt to get closer to 'master+feature1'
> > > testing from 'master+feature1+feature2' testing [yeah - its more like
> > > master+2/3 branches in next-tmp vs master+10/15 branches in next.]
> > >
> > > Also I'm restarting next-tmp from a clean master when merging new set
> > > of branches to test. And throwing away branches with problems - and
> > > retest only after it has fixes [This way a broken branch does not keep
> > > next-tmp broken until it gets fixed.]
> >
> >
> > I don't think we have the resources to run full tests on every branch one
> > at a time. Do we?
>
> We don't yet have this - but thats what I'm effectively doing manually
> with next-tmp [instead of 1 - its 2-3]
>
> Barry's thought here (I think) is - once we convert all the examples
> to the new test suite - it whould run much quicker - so might be able
> to a whole test suite on each branch.
>
> Or mabe have a way to have a few (fast) tests that can uncover most of
> the issues [aka travis-ci, pipelines] . Once the branch clears this -
> it can go into longer test queue (aka next equivalnet - but not next).
>
> None of this infrastructure currently exists.
>
> BTW: Ultimlately if you want to improve current next model - everyone
> has to do a 'make alltests DIFF=$PETSC_DIR/bin/petscdiff' for atleast
> one build that has relavent feature options enabled - before merging
> the feature branch to next.
>

This does not agree with my experience (I think) for bug finding. It tends
to
be other architectures (complex, 64-bit int, fortran) that fail for me,
which
means running that for many builds, and my laptop is just not capable, and
furthermore that is why we have a farm of machines.

   Matt


> Note - this doesn't have to be on ones laptop - it can be on es.mcs.
>
> Satish
>



-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener

https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20171111/3056e5a5/attachment.html>