[petsc-dev] snes_tutorials-ex19_cuda_1

[Satish Balay]

https://gitlab.com/petsc/petsc/pipelines/132414153/builds

this pipeline had 1 rerun for  linux-cuda-double  and 5 reruns for linux-c-exodus-dbg

I've reverted 2655 for now. We can revert this once the issue is resolved.

Satish

On Thu, 2 Apr 2020, Satish Balay via petsc-dev wrote:

> Perhaps we should revert this merge - so that the CI is stabilized for now. 
> 
> Satish
> 
> On Thu, 2 Apr 2020, Junchao Zhang wrote:
> 
> > Seems caused by MR 2655
> > <https://gitlab.com/petsc/petsc/-/merge_requests/2655>.  I reverted it and
> > tested in CI several times and the error did not appear. Let's assume the
> > MR has a bug. I am looking into it.
> > 
> > --Junchao Zhang
> > 
> > 
> > On Thu, Apr 2, 2020 at 10:58 AM Satish Balay <balay at mcs.anl.gov> wrote:
> > 
> > > That was a different error. This does keep coming up occasionally.
> > > https://gitlab.com/petsc/petsc/-/issues/360#note_250063306
> > >
> > > The current issue is:
> > > https://gitlab.com/petsc/petsc/-/issues/360#note_314185490
> > >
> > > Satish
> > >
> > >
> > > On Thu, 2 Apr 2020, Karl Rupp wrote:
> > >
> > > > The fluctuations in this example have been fixed a few months ago; the
> > > issue
> > > > was the use of multiple streams instead of a single one. Maybe
> > > additional CUDA
> > > > streams have been reintroduced recently?
> > > >
> > > > Best regards,
> > > > Karli
> > > >
> > > >
> > > > On 4/2/20 5:02 AM, Junchao Zhang wrote:
> > > > > I could not reproduce it locally. Even in the CI, it is random.
> > > > >
> > > > > --Junchao Zhang
> > > > >
> > > > >
> > > > > On Wed, Apr 1, 2020 at 7:47 PM Matthew Knepley <knepley at gmail.com
> > > > > <mailto:knepley at gmail.com>> wrote:
> > > > >
> > > > >     I saw Satish talking about this on the CI Tracker MR.
> > > > >
> > > > >         Matt
> > > > >
> > > > >     On Wed, Apr 1, 2020 at 8:36 PM Lisandro Dalcin <dalcinl at gmail.com
> > > > >     <mailto:dalcinl at gmail.com>> wrote:
> > > > >
> > > > >         Well, my request will not fix the problem:
> > > > >         https://gitlab.com/petsc/petsc/-/jobs/495147366#L5231
> > > > >
> > > > >         On Thu, 2 Apr 2020 at 03:26, Lisandro Dalcin <
> > > dalcinl at gmail.com
> > > > >         <mailto:dalcinl at gmail.com>> wrote:
> > > > >
> > > > >             Can anyone messing with CPUs please update test
> > > > >             snes_tutorials-ex19_cuda_1 to use -ksp_monitor_short and
> > > > >             update its output with REPLACE=1 ?
> > > > >
> > > > >             Please do it in maint, or cherry-pick if already fixed in
> > > > >             master.
> > > > >
> > > > >             Regards,
> > > > >
> > > > >             --
> > > > >             Lisandro Dalcin
> > > > >             ============
> > > > >             Research Scientist
> > > > >             Extreme Computing Research Center (ECRC)
> > > > >             King Abdullah University of Science and Technology (KAUST)
> > > > >             http://ecrc.kaust.edu.sa/
> > > > >
> > > > >
> > > > >
> > > > >         --
> > > > >         Lisandro Dalcin
> > > > >         ============
> > > > >         Research Scientist
> > > > >         Extreme Computing Research Center (ECRC)
> > > > >         King Abdullah University of Science and Technology (KAUST)
> > > > >         http://ecrc.kaust.edu.sa/
> > > > >
> > > > >
> > > > >
> > > > >     --
> > > > >     What most experimenters take for granted before they begin their
> > > > >     experiments is infinitely more interesting than any results to
> > > which
> > > > >     their experiments lead.
> > > > >     -- Norbert Wiener
> > > > >
> > > > >     https://www.cse.buffalo.edu/~knepley/
> > > > >     <http://www.cse.buffalo.edu/~knepley/>
> > > > >
> > > >
> > > >
> > >
> > 
>