[Nek5000-users] Nek: abnormal termination by signal 11

Fri Mar 10 08:08:54 CST 2017

Do you restart from a single field file? 
Was the restart file generated on a big-endian system like BGQ?
Are you using MPIIO? 

We read the data in chunks so there is no need to experiment with lelt. However, it might be still a good idea. You never know ;)
What's the memory footprint before the crash? Just use getmaxrss() to print out the max persistent memory.

Can you try again with today's master?

Cheers,
Stefan

-----Original message-----
> From:nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
> Sent: Friday 10th March 2017 14:13
> To: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] Nek: abnormal termination by signal 11
> 
> 
> Hi Jan,
> 
> By how much are you reducing your resolution?
> May I suggest compiling with a larger lelt and then trying ?
> 
> I think that what is happening when you reduce lx1 is that there is not enough room
> to hold the restart field... increasing lelt will alleviate this problem.
> 
> Paul
> 
> 
> ________________________________________
> From: nek5000-users-bounces at lists.mcs.anl.gov [nek5000-users-bounces at lists.mcs.anl.gov] on behalf of nek5000-users at lists.mcs.anl.gov [nek5000-users at lists.mcs.anl.gov]
> Sent: Friday, March 10, 2017 6:45 AM
> To: nek5000-users at lists.mcs.anl.gov
> Subject: Re: [Nek5000-users] Nek: abnormal termination by signal 11
> 
> Hello,
> some more testing showed that the error is actually not dependent on the deformation of the grid in usrdat2. It occurs if I use a restart file and at the same time use a lower order lx1 etc. in SIZE. I sometimes do that when I suspect the resolution is too high. Increasing lx1 on the other hand is OK. Again, this error does not occur on an Intel system. The pipe/stenosis example is fine when reducing lx too, but it is also much smaller than my cases (around 10^4 Elements). I will try too find a small case that still reproduces the error.
> Jan
> 
> > Am 07.03.2017 um 14:21 schrieb nek5000-users at lists.mcs.anl.gov:
> >
> > I don't think his problem is related to compiler settings. The default should just work.
> > Also it looks like it's a case specific problem as the NekExamples run fine.
> >
> > -----Original message-----
> >> From:nek5000-users at lists.mcs.anl.gov <nek5000-users at lists.mcs.anl.gov>
> >> Sent: Tuesday 7th March 2017 14:18
> >> To: nek5000-users at lists.mcs.anl.gov
> >> Subject: Re: [Nek5000-users] Nek: abnormal termination by signal 11
> >>
> >>
> >> Hi Jan,
> >>
> >> I don't know if this helps, but this is the makenek script I use on BGQ.
> >>
> >> Paul
> >>
> >> ________________________________________
> >> From: nek5000-users-bounces at lists.mcs.anl.gov [nek5000-users-bounces at lists.mcs.anl.gov] on behalf of nek5000-users at lists.mcs.anl.gov [nek5000-users at lists.mcs.anl.gov]
> >> Sent: Tuesday, March 07, 2017 1:31 AM
> >> To: nek5000-users at lists.mcs.anl.gov
> >> Subject: Re: [Nek5000-users] Nek: abnormal termination by signal 11
> >>
> >> Hello Paul,
> >> I’m working on the Juqueen IBM BlueGene/Q in Juelich. (http://www.fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JUQUEEN/Configuration/Configuration_node.html)
> >> Jan
> >>> Am 07.03.2017 um 04:15 schrieb nek5000-users at lists.mcs.anl.gov:
> >>>
> >>>
> >>> Hi Jan,
> >>>
> >>> Sorry if you've already answered this question Which machine are you running on?
> >>>
> >>> Paul
> >>> ________________________________________
> >>> From: nek5000-users-bounces at lists.mcs.anl.gov [nek5000-users-bounces at lists.mcs.anl.gov] on behalf of nek5000-users at lists.mcs.anl.gov [nek5000-users at lists.mcs.anl.gov]
> >>> Sent: Monday, March 06, 2017 4:31 AM
> >>> To: nek5000-users at lists.mcs.anl.gov
> >>> Subject: Re: [Nek5000-users] Nek: abnormal termination by signal 11
> >>>
> >>> Hello again,
> >>> after some more trying around I got some of my cases to work again by doing a complete reset of the simulations - I was slowly increasing a forcing parameter, and by starting at zero again it seemed to work this time. Other simulations are still failing, even with the new Nek version 17.0. I directly copy the case folders between the clusters and only change the compilers in makenek and number of processors in SIZE, but the cases still work on Intel but fail on IBM.
> >>>
> >>>> Am 01.03.2017 um 11:18 schrieb nek5000-users at lists.mcs.anl.gov:
> >>>>
> >>>> The stenosis example is working fine with a restart. I found an older case from a few months ago that is still working fine, so I copied its .usr file to one of the new cases and it still failed, so I am suspecting that the error might be related to the grid. The parameters in the rea file are also largely the same.
> >>>>
> >>>> _______________________________________________
> >>>> Nek5000-users mailing list
> >>>> Nek5000-users at lists.mcs.anl.gov
> >>>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>>
> >>> _______________________________________________
> >>> Nek5000-users mailing list
> >>> Nek5000-users at lists.mcs.anl.gov
> >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>> _______________________________________________
> >>> Nek5000-users mailing list
> >>> Nek5000-users at lists.mcs.anl.gov
> >>> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>
> >> _______________________________________________
> >> Nek5000-users mailing list
> >> Nek5000-users at lists.mcs.anl.gov
> >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >> _______________________________________________
> >> Nek5000-users mailing list
> >> Nek5000-users at lists.mcs.anl.gov
> >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> >>
> > _______________________________________________
> > Nek5000-users mailing list
> > Nek5000-users at lists.mcs.anl.gov
> > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> 
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> _______________________________________________
> Nek5000-users mailing list
> Nek5000-users at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users
> 