From nek5000-users at lists.mcs.anl.gov Sat Oct 2 11:43:41 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 2 Oct 2010 11:43:41 -0500 (CDT) Subject: [Nek5000-users] Hpts Routine Not outputting data Message-ID: <1674050103.11371286037821917.JavaMail.root@neo-mail-3> Hi, I recently was having trouble with the hpts routine. It was giving me NanQ for all points at all steps, this was on repo 548. I was able to get it to work before, but moved my case ?to another computer and this was the result. I tried updating to the latest repo (568), but now nothing is dumped to hpts.out, not even the header. - Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Oct 4 00:12:19 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 00:12:19 -0500 (CDT) Subject: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? In-Reply-To: <1540476225.271651286169027739.JavaMail.root@neo-mail-3> Message-ID: <2067424271.272031286169139338.JavaMail.root@neo-mail-3> Hi, I can't get a run to compile, both with an old repo and the latest repo . I get the following error message during the compling :? It says something about waiting for unfinished jobs, and the a code generation return code. I don't know what it is talking about, but I have tried everything I know of and it still won't compile. I did quit a previous run using echo -1 which failed to fully write the f file, and this issue occurred right after,?I don't know if this is related or not. As anyone had this problem before or know how to solve it?? I was able to get this case to run but now it is telling me this and I am stumped. This is the portion of the compile log where the error occurs and it quits. - Michael 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp /ipaD2JMab". 1501-511? Compilation failed for file mvmesh .f. gmake : *** [obj/ mvmesh .o] Error 1 gmake : *** Waiting for unfinished jobs.... "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp / ipaNHnCUa ". 1501-511? Compilation failed for file ssolv .f. gmake : *** [obj/ ssolv .o] Error 1 "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-016 (W) WARNING: Compiler problem occurred while compiling uzawa : Cannot exceed the user or group quota.. "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-034 (S) Cannot create object file. 1586-346 (U) An error occurred during code generation.? The code generation return code was 1. 1501-511? Compilation failed for file navier1.f. gmake : *** [obj/navier1.o] Error 1 -bash-3.00$ -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Oct 4 03:16:08 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 03:16:08 -0500 (CDT) Subject: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? In-Reply-To: <2067424271.272031286169139338.JavaMail.root@neo-mail-3> References: <2067424271.272031286169139338.JavaMail.root@neo-mail-3> Message-ID: Hi Michael, Can you compile any other code w/ the same procedure? Paul On Mon, 4 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > > > > Hi, > > > > I can't get a run to compile, both with an old repo and the latest repo . I get the following error message during the compling :?? It says something about waiting for unfinished jobs, and the a code generation return code. I don't know what it is talking about, but I have tried everything I know of and it still won't compile. I did quit a previous run using echo -1 which failed to fully write the f file, and this issue occurred right after,??I don't know if this is related or not. As anyone had this problem before or know how to solve it??? I was able to get this case to run but now it is telling me this and I am stumped. This is the portion of the compile log where the error occurs and it quits. > > > > - Michael > > > > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp /ipaD2JMab". > 1501-511?? Compilation failed for file mvmesh .f. > gmake : *** [obj/ mvmesh .o] Error 1 > gmake : *** Waiting for unfinished jobs.... > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp / ipaNHnCUa ". > 1501-511?? Compilation failed for file ssolv .f. > gmake : *** [obj/ ssolv .o] Error 1 > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.?? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-016 (W) WARNING: Compiler problem occurred while compiling uzawa : Cannot exceed the user or group quota.. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-034 (S) Cannot create object file. > 1586-346 (U) An error occurred during code generation.?? The code generation return code was 1. > 1501-511?? Compilation failed for file navier1.f. > gmake : *** [obj/navier1.o] Error 1 > -bash-3.00$ > From nek5000-users at lists.mcs.anl.gov Mon Oct 4 07:24:50 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 06:24:50 -0600 (GMT-06:00) Subject: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? In-Reply-To: Message-ID: <1853337045.638841286195090911.JavaMail.root@zimbra.anl.gov> Michael: It looks like the compiler cannot write some temporary files to the disk. I am pretty sure this problem is unrelated to Nek and you system has somehow a problem. -Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: Nekton User List Sent: Mon, 4 Oct 2010 02:16:08 -0600 (GMT-06:00) Subject: Re: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? Hi Michael, Can you compile any other code w/ the same procedure? Paul On Mon, 4 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > > > > Hi, > > > > I can't get a run to compile, both with an old repo and the latest repo . I get the following error message during the compling :? It says something about waiting for unfinished jobs, and the a code generation return code. I don't know what it is talking about, but I have tried everything I know of and it still won't compile. I did quit a previous run using echo -1 which failed to fully write the f file, and this issue occurred right after,?I don't know if this is related or not. As anyone had this problem before or know how to solve it?? I was able to get this case to run but now it is telling me this and I am stumped. This is the portion of the compile log where the error occurs and it quits. > > > > - Michael > > > > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp /ipaD2JMab". > 1501-511? Compilation failed for file mvmesh .f. > gmake : *** [obj/ mvmesh .o] Error 1 > gmake : *** Waiting for unfinished jobs.... > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp / ipaNHnCUa ". > 1501-511? Compilation failed for file ssolv .f. > gmake : *** [obj/ ssolv .o] Error 1 > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-016 (W) WARNING: Compiler problem occurred while compiling uzawa : Cannot exceed the user or group quota.. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-034 (S) Cannot create object file. > 1586-346 (U) An error occurred during code generation.? The code generation return code was 1. > 1501-511? Compilation failed for file navier1.f. > gmake : *** [obj/navier1.o] Error 1 > -bash-3.00$ > From nek5000-users at lists.mcs.anl.gov Mon Oct 4 07:37:16 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 07:37:16 -0500 (CDT) Subject: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? In-Reply-To: <1251847581.300111286195782359.JavaMail.root@neo-mail-3> Message-ID: <922348293.300211286195836520.JavaMail.root@neo-mail-3> Hi Stefan, Paul, Yes I was able to resolve the issue; I didn't realize my disk space was full and received the?error message around the time I did?something else I thought caused it. Thanks for looking at it though. - Michael ? ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Monday, October 4, 2010 7:24:50 AM GMT -06:00 US/Canada Central Subject: Re: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? Michael: It looks like the compiler cannot write some temporary files to the disk. I am pretty sure this problem is unrelated to Nek and you system has somehow a problem. -Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: Nekton User List Sent: Mon, 4 Oct 2010 02:16:08 -0600 (GMT-06:00) Subject: Re: [Nek5000-users] Compiling issue, waiting for unfinished jobs...?? Hi Michael, Can you compile any other code w/ the same procedure? Paul On Mon, 4 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > > > > Hi, > > > > I can't get a run to compile, both with an old repo and the latest repo . I get the following error message during the compling :? It says something about waiting for unfinished jobs, and the a code generation return code. I don't know what it is talking about, but I have tried everything I know of and it still won't compile. I did quit a previous run using echo -1 which failed to fully write the f file, and this issue occurred right after,?I don't know if this is related or not. As anyone had this problem before or know how to solve it?? I was able to get this case to run but now it is telling me this and I am stumped. This is the portion of the compile log where the error occurs and it quits. > > > > - Michael > > > > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp /ipaD2JMab". > 1501-511? Compilation failed for file mvmesh .f. > gmake : *** [obj/ mvmesh .o] Error 1 > gmake : *** Waiting for unfinished jobs.... > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > 1586-444 (U) An error occurred while writing to file "/scratch/cmm7686/ tmp / ipaNHnCUa ". > 1501-511? Compilation failed for file ssolv .f. > gmake : *** [obj/ ssolv .o] Error 1 > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-036 (I) The NOSTRICT option (default at OPT(3)) has the potential to alter the semantics of a program.? Please refer to documentation on the STRICT/ NOSTRICT option for more information. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-016 (W) WARNING: Compiler problem occurred while compiling uzawa : Cannot exceed the user or group quota.. > "/home/cmm7686/nek5_548/trunk/ nek /navier1.f", 1500-034 (S) Cannot create object file. > 1586-346 (U) An error occurred during code generation.? The code generation return code was 1. > 1501-511? Compilation failed for file navier1.f. > gmake : *** [obj/navier1.o] Error 1 > -bash-3.00$ > _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Oct 4 09:23:25 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 08:23:25 -0600 (GMT-06:00) Subject: [Nek5000-users] Hpts Routine Not outputting data In-Reply-To: <1674050103.11371286037821917.JavaMail.root@neo-mail-3> Message-ID: <910909601.643361286202205683.JavaMail.root@zimbra.anl.gov> Please send in your case. -Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: "Nekton User List" Sent: Saturday, October 2, 2010 11:43:41 AM GMT -06:00 US/Canada Central Subject: [Nek5000-users] Hpts Routine Not outputting data Hi, I recently was having trouble with the hpts routine. It was giving me NanQ for all points at all steps, this was on repo 548. I was able to get it to work before, but moved my case to another computer and this was the result. I tried updating to the latest repo (568), but now nothing is dumped to hpts.out, not even the header. - Michael _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Mon Oct 4 09:35:26 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 09:35:26 -0500 (CDT) Subject: [Nek5000-users] Hpts Routine Not outputting data In-Reply-To: <910909601.643361286202205683.JavaMail.root@zimbra.anl.gov> Message-ID: <1351291216.322511286202926791.JavaMail.root@neo-mail-3> Hi Stefan, I am using a restart on this run and the two restart files cause my .tgz to be nearly 200MB. Is there an easier way to send it instead of through mail, or do you not need the restart fld files? ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Monday, October 4, 2010 9:23:25 AM GMT -06:00 Guadalajara / Mexico City / Monterrey Subject: Re: [Nek5000-users] Hpts Routine Not outputting data Please send in your case. -Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: "Nekton User List" Sent: Saturday, October 2, 2010 11:43:41 AM GMT -06:00 US/Canada Central Subject: [Nek5000-users] Hpts Routine Not outputting data Hi, I recently was having trouble with the hpts routine. It was giving me NanQ for all points at all steps, this was on repo 548. I was able to get it to work before, but moved my case to another computer and this was the result. I tried updating to the latest repo (568), but now nothing is dumped to hpts.out, not even the header. - Michael _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Mon Oct 4 09:37:38 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 08:37:38 -0600 (GMT-06:00) Subject: [Nek5000-users] Hpts Routine Not outputting data In-Reply-To: <1351291216.322511286202926791.JavaMail.root@neo-mail-3> Message-ID: <1587216126.644551286203058628.JavaMail.root@zimbra.anl.gov> I think we don't need the restart data. Just send the case files. Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Monday, October 4, 2010 9:35:26 AM GMT -06:00 US/Canada Central Subject: Re: [Nek5000-users] Hpts Routine Not outputting data Hi Stefan, I am using a restart on this run and the two restart files cause my .tgz to be nearly 200MB. Is there an easier way to send it instead of through mail, or do you not need the restart fld files? ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Monday, October 4, 2010 9:23:25 AM GMT -06:00 Guadalajara / Mexico City / Monterrey Subject: Re: [Nek5000-users] Hpts Routine Not outputting data Please send in your case. -Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: "Nekton User List" Sent: Saturday, October 2, 2010 11:43:41 AM GMT -06:00 US/Canada Central Subject: [Nek5000-users] Hpts Routine Not outputting data Hi, I recently was having trouble with the hpts routine. It was giving me NanQ for all points at all steps, this was on repo 548. I was able to get it to work before, but moved my case to another computer and this was the result. I tried updating to the latest repo (568), but now nothing is dumped to hpts.out, not even the header. - Michael _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Mon Oct 4 09:38:09 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 4 Oct 2010 09:38:09 -0500 (CDT) Subject: [Nek5000-users] Hpts Routine Not outputting data In-Reply-To: <1587216126.644551286203058628.JavaMail.root@zimbra.anl.gov> References: <1587216126.644551286203058628.JavaMail.root@zimbra.anl.gov> Message-ID: good point... On Mon, 4 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > I think we don't need the restart data. Just send the case files. > Stefan > > ----- Original Message ----- > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Sent: Monday, October 4, 2010 9:35:26 AM GMT -06:00 US/Canada Central > Subject: Re: [Nek5000-users] Hpts Routine Not outputting data > > > > Hi Stefan, > > I am using a restart on this run and the two restart files cause my .tgz to be nearly 200MB. Is there an easier way to send it instead of through mail, or do you not need the restart fld files? > > ----- Original Message ----- > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Sent: Monday, October 4, 2010 9:23:25 AM GMT -06:00 Guadalajara / Mexico City / Monterrey > Subject: Re: [Nek5000-users] Hpts Routine Not outputting data > > Please send in your case. > -Stefan > > > ----- Original Message ----- > From: nek5000-users at lists.mcs.anl.gov > To: "Nekton User List" > Sent: Saturday, October 2, 2010 11:43:41 AM GMT -06:00 US/Canada Central > Subject: [Nek5000-users] Hpts Routine Not outputting data > > > > > > Hi, > > > > I recently was having trouble with the hpts routine. It was giving me NanQ for all points at > > all steps, this was on repo 548. I was able to get it to work before, but moved my case > > to another computer and this was the result. I tried updating to the latest repo (568), but > > now nothing is dumped to hpts.out, not even the header. > > > > - Michael > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Fri Oct 8 09:48:16 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 8 Oct 2010 09:48:16 -0500 (CDT) Subject: [Nek5000-users] Planar averaging in 2D In-Reply-To: References: Message-ID: Hi David, The attached routine was written for planar averaging in y -- Have a look - it should in principle work in 2D as well. Note that it's not been tested in 2D or 3D as far as I know. I wrote it for someone a week or two ago (with the same caveat). Am posting to the user list in case someone knows if this is working or has been tested/fixed. Paul On Fri, 8 Oct 2010, Aleksandr Obabko wrote: > Hi David, > > You are correct -- planar_avarge_z has to be modified. I'm finishing the > report today so after it I can make a pass at the routine. > > Best, > Aleks > > > On Fri, 8 Oct 2010, David Goluskin wrote: > >> Hi Aleks, >> >> A few months ago, you told me how to get horizontally averaged temperature >> profiles using the planar_average_z routine. To get horizontal averages >> for >> a 2D run, I am guessing I need to either modify the planar averaging >> routine >> or change the vertical coordinate form y to z, which I don`t know how to >> do. What do you suggest? >> >> Thank you, >> >> David >> > -------------- next part -------------- c----------------------------------------------------------------------- subroutine planar_average_y(ua,u2,u,w1,w2) c c Compute r-s planar average of quantity u() and u^2() c include 'SIZE' include 'GEOM' include 'PARALLEL' include 'WZ' include 'ZPER' c real ua(ny1,nely),u2(ny1,nely) ! Output: < u > and < u^2 > real u(nx1,ny1,nz1,nelv) ! Input real w1(ny1,nely),w2(ny1,nely) ! work arrays integer e,eg,ex,ey,ez ny = ny1*nely call rzero(ua,ny) call rzero(u2,ny) call rzero(w1,ny) do e=1,nelt eg = lglel(e) call get_exyz(ex,ey,ez,nelx,nely,nelz) do k=1,nz1 do i=1,nx1 do j=1,ny1 yy = (1.-zgm1(j,2))/2. ! = 1 for j=1, = 0 for j=ny1 aa = yy*area(i,1,1,e) + (1-yy)*area(i,1,3,e) ! wgtd jacobian, fc 1&3 w1(j,ey) = w1(j,ey) + aa ua(j,ey) = ua(j,ey) + aa*u(i,j,k,e) u2(j,ey) = u2(j,ey) + aa*u(i,j,k,e)**2 enddo enddo enddo enddo call gop(ua,w2,'+ ',ny) call gop(u2,w2,'+ ',ny) call gop(w1,w2,'+ ',ny) do i=1,ny ua(i,1) = ua(i,1) / w1(i,1) ! Normalize u2(i,1) = u2(i,1) / w1(i,1) ! Normalize enddo return end c----------------------------------------------------------------------- From nek5000-users at lists.mcs.anl.gov Fri Oct 8 18:16:48 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 8 Oct 2010 18:16:48 -0500 Subject: [Nek5000-users] Planar averaging in 2D In-Reply-To: References: Message-ID: Hi Paul, Does this routine work for non-tensor product domains ? I had been wanting to do spatial averaging in nek for non-tensor product domains, but haven't found a way yet. Regards Shriram Jagannathan On 8 October 2010 09:48, wrote: > > Hi David, > > The attached routine was written for planar averaging in y -- > > Have a look - it should in principle work in 2D as well. > Note that it's not been tested in 2D or 3D as far as I know. > I wrote it for someone a week or two ago (with the same > caveat). Am posting to the user list in case someone knows > if this is working or has been tested/fixed. > > Paul > > On Fri, 8 Oct 2010, Aleksandr Obabko wrote: > > Hi David, >> >> You are correct -- planar_avarge_z has to be modified. I'm finishing the >> report today so after it I can make a pass at the routine. >> >> Best, >> Aleks >> >> >> On Fri, 8 Oct 2010, David Goluskin wrote: >> >> Hi Aleks, >>> >>> A few months ago, you told me how to get horizontally averaged >>> temperature >>> profiles using the planar_average_z routine. To get horizontal averages >>> for >>> a 2D run, I am guessing I need to either modify the planar averaging >>> routine >>> or change the vertical coordinate form y to z, which I don`t know how to >>> do. What do you suggest? >>> >>> Thank you, >>> >>> David >>> >>> > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Oct 8 19:12:00 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 8 Oct 2010 19:12:00 -0500 (CDT) Subject: [Nek5000-users] Planar averaging in 2D In-Reply-To: References: Message-ID: For non-tensor products you would need to use the interpolation routines. Paul On Fri, 8 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > Hi Paul, > > Does this routine work for non-tensor product domains ? I had been wanting > to do spatial averaging in nek for non-tensor product domains, but haven't > found a way yet. > > Regards > Shriram Jagannathan > > > On 8 October 2010 09:48, wrote: > >> >> Hi David, >> >> The attached routine was written for planar averaging in y -- >> >> Have a look - it should in principle work in 2D as well. >> Note that it's not been tested in 2D or 3D as far as I know. >> I wrote it for someone a week or two ago (with the same >> caveat). Am posting to the user list in case someone knows >> if this is working or has been tested/fixed. >> >> Paul >> >> On Fri, 8 Oct 2010, Aleksandr Obabko wrote: >> >> Hi David, >>> >>> You are correct -- planar_avarge_z has to be modified. I'm finishing the >>> report today so after it I can make a pass at the routine. >>> >>> Best, >>> Aleks >>> >>> >>> On Fri, 8 Oct 2010, David Goluskin wrote: >>> >>> Hi Aleks, >>>> >>>> A few months ago, you told me how to get horizontally averaged >>>> temperature >>>> profiles using the planar_average_z routine. To get horizontal averages >>>> for >>>> a 2D run, I am guessing I need to either modify the planar averaging >>>> routine >>>> or change the vertical coordinate form y to z, which I don`t know how to >>>> do. What do you suggest? >>>> >>>> Thank you, >>>> >>>> David >>>> >>>> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >> >> > From nek5000-users at lists.mcs.anl.gov Tue Oct 12 09:29:00 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 12 Oct 2010 09:29:00 -0500 Subject: [Nek5000-users] Reminder - please register for the First Nek5000 Workshop ASAP! Message-ID: <9C48D676-9E34-4742-B663-85D677A0D616@mcs.anl.gov> Dear Nek5000 Users: We are pleased to invite you to attend the first Nek5000 Workshop, to be held at Argonne National Laboratory on December 9th and 10th, 2010. The intent is to bring developers and users of Nek5000 together to promote collaborative activities, exchange information, and share experiences in areas of common interest. The schedule is under development, but we are currently planning to have technical sessions on both days, December 9th and December 10th, including hands-on sessions in the use of Nek5000, VisIt, and Cubit. One objective of the workshop is to assist users in defining their simulation problems and to address technical issues that they may face. Attendees are invited to contribute short (15 min.) presentations on their applications of Nek5000. Contributions should consist of an abstract (1-2 pages) and presentation slides. Please note that, as this is the first users meeting, we are uncertain of anticipated interest and may, therefore, cancel the meeting if we do not meet a critical attendance limit. It is imperative that you register by the deadline if you plan to attend, so that suitable arrangements can be made at Argonne National Laboratory. REGISTRATION DEADLINE: October 15, 2010 Please fill out the information in the link listed below. This information will be used for procurement of a gate pass and as your RSVP for the workshop. www.cels.anl.gov/about/visitor_registration.php please type Paul Fischer in the "Who is hosting your visit" and chose Mary Dzielski, as the "Administrative Contact" Recommendation for hotel accommodations: Argonne's Guest House: $70 + tax, 1-800-632-8990 http://www.anlgh.org Helpful link for visiting Argonne: http://www.anl.gov/Administration/visit.html Any questions regarding travel arrangements, please contact Mary Dzielski (dzielski at mcs.anl.gov). Questions regarding the workshop, please contact Paul Fischer (fischer at mcs.anl.gov) or Stefan Kerkemeier (stefanke at mcs.anl.gov) We look forward to your attendance! Sincerely, Nek5000 Development Team -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Tue Oct 12 11:45:41 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 12 Oct 2010 12:45:41 -0400 Subject: [Nek5000-users] problem with DNS of heat transfer in a channel Message-ID: Hello, Paul, Thank you for your answer for the same problem last month. But I still could not figure out this issue. (Domain:6.4/2.0/3.2, Resolution: 12*8*12 elements with 8th order, turbulent Re_tau=178.12, Pr=0.71) , piece of my *.rea file ***************************************************************************** 1.00000 DENSITY -178.12 Re_b 0.000000E+00 BETAG 0.00000 GTHETA 0.000000E+00 PGRADX 0.000000E+00 FLOWRATE 1.0 RHOCP -126.47 CONDUCT ***************************************************************************** my *usr file ***************************************************************************** subroutine userq (ix,iy,iz,ieg) include 'SIZE' include 'TOTAL' include 'NEKUSE' ie = gllel(ieg) qvol = vx(ix,iy,iz,ie)/15.771 source = 0.0 return end ***************************************************************************** Here 15.71 is averaged mean velocity over channel section. After a long time running (dimensionless time > 350 ) I compared the result , the velocity profile looks good. But the temperature is much higher than Kasagi's result. I'm wondering whether the parameters above are correct ? I attached the plot of mean velocity and mean temperature. Thank you again -- Zhicheng Wang -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: means.pdf Type: application/pdf Size: 6991 bytes Desc: not available URL: From nek5000-users at lists.mcs.anl.gov Wed Oct 13 04:58:37 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 13 Oct 2010 11:58:37 +0200 Subject: [Nek5000-users] Extrude an axisymmetric flow onto a fully thee-dimensional mesh Message-ID: <4CB582CD.9040409@gmail.com> Hi Nek's :-) , I have computed an axisymmetric flow using the IFAXIS option in Nek. I would like now to extrude / reconstruct it onto a fully three-dimensional mesh to perform another calculation using it as a base flow. Is there any routine already existing enabling me to do so? If not, a few hints on how I should do it would be highly appreciated :-) Best regards, JC From nek5000-users at lists.mcs.anl.gov Wed Oct 13 07:23:37 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 13 Oct 2010 07:23:37 -0500 (CDT) Subject: [Nek5000-users] Extrude an axisymmetric flow onto a fully thee-dimensional mesh In-Reply-To: <4CB582CD.9040409@gmail.com> References: <4CB582CD.9040409@gmail.com> Message-ID: Hi JC, You _might_ be able to make it work by using the grid-to-grid intrepolation described here: http://nek5000.mcs.anl.gov/index.php/Data_processing_example As I think about it, what you would need to do is something like new (3D) geometry - .usr file: subroutine userchk n=nx1*ny1*nz1*nelv do i=1,n x=xm1(i,1,1,1) ! assume x is the axis and y-z is in cross-plane y=ym1(i,1,1,1) z=zm1(i,1,1,1) rr = y*y+z*z if (rr.gt.0) rr=sqrt(rr) ym1(i,1,1,1) = rr zm1(i,1,1,1) = 0 enddo ifxyo=.true. call outpost(vx,vy,vz,pr,t,'g2d') call exitt This would write a file with x,y in the meridional plane. Interpolate your old solution onto these points. Not certain if g2g can handle reading a 3D field file in a 2D case -- probably not. In that case, we just use the interpolator and generate our own file i/o handler. I have some things sitting around but they are a bit out of step with the current interpolation interface. Paul On Wed, 13 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > Hi Nek's :-) , > > I have computed an axisymmetric flow using the IFAXIS option in Nek. I would > like now to extrude / reconstruct it onto a fully three-dimensional mesh to > perform another calculation using it as a base flow. > Is there any routine already existing enabling me to do so? If not, a few > hints on how I should do it would be highly appreciated :-) > > Best regards, > JC > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Fri Oct 15 05:27:43 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 15 Oct 2010 12:27:43 +0200 Subject: [Nek5000-users] Extrude an axisymmetric flow onto a fully thee-dimensional mesh In-Reply-To: References: <4CB582CD.9040409@gmail.com> Message-ID: Hi Paul, I have just tried with a Poiseuille test case, using both code snipplets from your email and the link you gave me, but it does not seem to work. Here is what I get from the logfile: call userchk grid-to-grid interpolation NONSTD HDR, parse_hdr, abort. call exitt: dying ... Is that the output you were expecting when writing 'Not certain if g2g can handle reading a 3D field file in a 2D case -- probably not', or did I perhaps misdo something? JC -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Tue Oct 19 06:18:41 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Tue, 19 Oct 2010 13:18:41 +0200 Subject: [Nek5000-users] scaling of nek5000 on a Cray Message-ID: <4CBD7E91.70009@mech.kth.se> Hi nek5000-users & developers, I wonder if anyone has experience with the performance of nek5000 on a Cray ? In particular what scaling could one expect compared to e.g. a Blue Gene? We have a new Cray XT6m system, based on the AMD Opteron 12-core ?Magny-Cours? (2.1 GHz) processors and the Cray SeaStar2 interconnect technology. It consists of 11040 compute cores and 10 service cores. The scaling is not too encouraging (see attached pdf-files). The attached files show three different runs, and the scaling is apparently highly dependent on where in the torus the job happens to be. This is maybe ok, but is the leveling off after 2048 cores consistent with what people observe on a Cray ? Thanks for your suggestions and experience! Best regards, Johan -- Johan Malm Department of Mechanics, KTH SE-100 44, Stockholm, Sweden Phone: +46 8 7906876 E-mail: johan at mech.kth.se -------------- next part -------------- A non-text attachment was scrubbed... Name: scaling1.pdf Type: application/pdf Size: 3975 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: scaling3.pdf Type: application/pdf Size: 3948 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: scaling4.pdf Type: application/pdf Size: 3943 bytes Desc: not available URL: From nek5000-users at lists.mcs.anl.gov Wed Oct 20 01:56:05 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 20 Oct 2010 00:56:05 -0600 (GMT-06:00) Subject: [Nek5000-users] scaling of nek5000 on a Cray In-Reply-To: <4CBD7E91.70009@mech.kth.se> Message-ID: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> Hi Johan, we can definitely do better on Cray systems but I guess we need to do some fine tuning first. The BG architecture is quite different and you need much more grid points per core to scale on the Cray (4-8x more). - Can you post your logfiles? - Did you try to enable Nek's internal MPI profiling where you spend most of the communication cost? - Did you try to use an optimized MPI rank mapping? Is it possible to get an account on that machine to do some experiments? Cheers, Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Tue, 19 Oct 2010 05:18:41 -0600 (GMT-06:00) Subject: [Nek5000-users] scaling of nek5000 on a Cray Hi nek5000-users & developers, I wonder if anyone has experience with the performance of nek5000 on a Cray ? In particular what scaling could one expect compared to e.g. a Blue Gene? We have a new Cray XT6m system, based on the AMD Opteron 12-core ?Magny-Cours? (2.1 GHz) processors and the Cray SeaStar2 interconnect technology. It consists of 11040 compute cores and 10 service cores. The scaling is not too encouraging (see attached pdf-files). The attached files show three different runs, and the scaling is apparently highly dependent on where in the torus the job happens to be. This is maybe ok, but is the leveling off after 2048 cores consistent with what people observe on a Cray ? Thanks for your suggestions and experience! Best regards, Johan -- Johan Malm Department of Mechanics, KTH SE-100 44, Stockholm, Sweden Phone: +46 8 7906876 E-mail: johan at mech.kth.se From nek5000-users at lists.mcs.anl.gov Wed Oct 20 02:22:09 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 20 Oct 2010 02:22:09 -0500 (CDT) Subject: [Nek5000-users] scaling of nek5000 on a Cray In-Reply-To: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> References: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> Message-ID: I'm hoping to include our ping-pong and mxm timers in a future source release so that we can get a handle on the platform parameters via a couple of quick calls from userchk. With luck, we'll have this in a few days. -- Paul On Wed, 20 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > Hi Johan, we can definitely do better on Cray systems but I guess we need to do some fine tuning first. The BG architecture is quite different and you need much more grid points per core to scale on the Cray (4-8x more). - Can you post your logfiles? - Did you try to enable Nek's internal MPI profiling where you spend most of the communication cost? - Did you try to use an optimized MPI rank mapping? Is it possible to get an account on that machine to do some experiments? Cheers, Stefan ----- Original Message ----- From: nek5000-users at lists.mcs.anl.gov To: nek5000-users at lists.mcs.anl.gov Sent: Tue, 19 Oct 2010 05:18:41 -0600 (GMT-06:00) Subject: [Nek5000-users] scaling of nek5000 on a Cray Hi nek5000-users & developers, I wonder if anyone has experience with the performance of nek5000 on a Cray ? In particular what scaling could one expect compared to e.g. a Blue Gene? We have a new Cray XT6m system, based on the AMD Opteron 12-core ?Magny-Cours? (2.1 GHz) processors and the Cray SeaStar2 interconnect technology. It consists of 11040 compute cores and 10 service cores. The scaling is not too encouraging (see attached pdf-files). The attached files show three different runs, and the scaling is apparently highly dependent on where in the torus the job happens to be. This is maybe ok, but is the leveling off after 2048 cores consistent with what people observe on a Cray ? Thanks for your suggestions and experience! Best regards, Johan -- Johan Malm Department of Mechanics, KTH SE-100 44, Stockholm, Sweden Phone: +46 8 7906876 E-mail: johan at mech.kth.se _______________________________________________ Nek5000-users mailing list Nek5000-users at lists.mcs.anl.gov https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Wed Oct 20 06:00:47 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Wed, 20 Oct 2010 13:00:47 +0200 Subject: [Nek5000-users] scaling of nek5000 on a Cray In-Reply-To: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> References: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> Message-ID: <4CBECBDF.9030709@mech.kth.se> Hello Stefan, Here is a (shortened) log file pertaining to a simulation on 4096 processors of `our' Cray XT6m. The case is as follows: * Nek5000, rev. 565 * 220248 elements in total, i.e. 54 elements/core * Restart from a nearly converged initial condition To your hints given below: How do I enable Nek's internal MPI profiling? We got a long list of runtime statistics and core-wise times; maybe the internal profiling was actually active... Could you give a brief explanation of the most important numbers, please? How can I invoke the optimized MPI rank mapping? Thank you in advance for some clarification! Lars (who collaborates with Johan) nek5000-users at lists.mcs.anl.gov wrote: > Hi Johan, > > we can definitely do better on Cray systems but I guess we need to do some fine tuning first. > The BG architecture is quite different and you need much more grid points per core to scale on the Cray (4-8x more). > > - Can you post your logfiles? > - Did you try to enable Nek's internal MPI profiling where you spend most of the communication cost? > - Did you try to use an optimized MPI rank mapping? > > Is it possible to get an account on that machine to do some experiments? > > > Cheers, > Stefan > > > ----- Original Message ----- > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Sent: Tue, 19 Oct 2010 05:18:41 -0600 (GMT-06:00) > Subject: [Nek5000-users] scaling of nek5000 on a Cray > > Hi nek5000-users & developers, > > I wonder if anyone has experience with the performance of nek5000 on a > Cray ? In particular what scaling could one expect compared to e.g. a > Blue Gene? > > We have a new Cray XT6m system, based on the AMD Opteron 12-core > ?Magny-Cours? (2.1 > GHz) processors and the Cray SeaStar2 interconnect technology. It > consists of 11040 compute cores and 10 service cores. > > The scaling is not too encouraging (see attached pdf-files). The > attached files show three different runs, and the scaling is apparently > highly dependent on where in the torus the job happens to be. This is > maybe ok, but is the leveling off after 2048 cores consistent with what > people observe on a Cray ? > > Thanks for your suggestions and experience! > > Best regards, > > Johan > > -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: cray_4096procs.out URL: From nek5000-users at lists.mcs.anl.gov Fri Oct 22 09:55:24 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 22 Oct 2010 16:55:24 +0200 Subject: [Nek5000-users] Steady state computation Message-ID: <4CC1A5DC.4030503@gmail.com> Hi Nek's :-) I would like to compute steady state solutions of the Navier-Stokes equations and use them as a base flow for stability calculations. Being base flows, these solutions are not necessarily the time-asymptotic steady solutions (if they exists). As a matter of fact, I was wondering if any steady-state solver was currently available in Nek or if I would have implement my own one? Sincerely, JC From nek5000-users at lists.mcs.anl.gov Fri Oct 22 12:50:07 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 22 Oct 2010 12:50:07 -0500 (CDT) Subject: [Nek5000-users] Steady state computation In-Reply-To: <4CC1A5DC.4030503@gmail.com> References: <4CC1A5DC.4030503@gmail.com> Message-ID: Hi, We have support for evolving perturbations on top of steady or unsteady base flows, but we do not have a steady-state NS solver, only a steady Stokes solver. I'd be willing to work with you if you would like to develop a steady NS variant... Paul On Fri, 22 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > Hi Nek's :-) > > I would like to compute steady state solutions of the Navier-Stokes equations > and use them as a base flow for stability calculations. Being base flows, > these solutions are not necessarily the time-asymptotic steady solutions (if > they exists). As a matter of fact, I was wondering if any steady-state solver > was currently available in Nek or if I would have implement my own one? > > > Sincerely, > JC > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > From nek5000-users at lists.mcs.anl.gov Sat Oct 23 08:13:09 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 23 Oct 2010 08:13:09 -0500 (CDT) Subject: [Nek5000-users] scaling of nek5000 on a Cray In-Reply-To: References: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> Message-ID: Hi Johan, I've added some performance tests to the repo (v. 570) There are a couple of calling options - I would suggest, in usrdat, within your .usr file to have iverbose = 1 ! fully verbose call platform_timer(iverbose) call exitti('timer done$',iverbose) and run this with a reasonably large 3D case on your parallel platform. If you can send us the output of this run off list we'll have a pretty good idea of what's going on. Thanks! Paul On Wed, 20 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > > I'm hoping to include our ping-pong and mxm timers in a future > source release so that we can get a handle on the platform > parameters via a couple of quick calls from userchk. With luck, > we'll have this in a few days. -- Paul > > > > On Wed, 20 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > >> Hi Johan, > > we can definitely do better on Cray systems but I guess we need to do some > fine tuning first. > The BG architecture is quite different and you need much more grid points per > core to scale on the Cray (4-8x more). > > - Can you post your logfiles? > - Did you try to enable Nek's internal MPI profiling where you spend most of > the communication cost? > - Did you try to use an optimized MPI rank mapping? > > Is it possible to get an account on that machine to do some experiments? > > > Cheers, > Stefan > > > ----- Original Message ----- > From: nek5000-users at lists.mcs.anl.gov > To: nek5000-users at lists.mcs.anl.gov > Sent: Tue, 19 Oct 2010 05:18:41 -0600 (GMT-06:00) > Subject: [Nek5000-users] scaling of nek5000 on a Cray > > Hi nek5000-users & developers, > > I wonder if anyone has experience with the performance of nek5000 on a > Cray ? In particular what scaling could one expect compared to e.g. a > Blue Gene? > > We have a new Cray XT6m system, based on the AMD Opteron 12-core > ???Magny-Cours??? (2.1 > GHz) processors and the Cray SeaStar2 interconnect technology. It > consists of 11040 compute cores and 10 service cores. > > The scaling is not too encouraging (see attached pdf-files). The > attached files show three different runs, and the scaling is apparently > highly dependent on where in the torus the job happens to be. This is > maybe ok, but is the leveling off after 2048 cores consistent with what > people observe on a Cray ? > > Thanks for your suggestions and experience! > > Best regards, > > Johan > > -- > Johan Malm > Department of Mechanics, KTH > SE-100 44, Stockholm, Sweden > Phone: +46 8 7906876 > E-mail: johan at mech.kth.se > > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users From nek5000-users at lists.mcs.anl.gov Mon Oct 25 05:39:41 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 25 Oct 2010 12:39:41 +0200 Subject: [Nek5000-users] Steady state computation In-Reply-To: References: <4CC1A5DC.4030503@gmail.com> Message-ID: <4CC55E6D.90204@gmail.com> Hello Paul, I would personally be highly interested into developing such a steady NS solver. However as I've just started my PhD a few weeks ago I should discuss with my two supervisors first. Base flow computation being absolutely necessary in linear stability calculations I will eventually have to develop such a solver anyway sooner or later. I'll re-contact you as soon as this preliminary discussion with them is done. Sincerely, JC. PS: You may perhaps know one of them: Emmanuel Leriche from the University of Saint-Etienne (France). He's been working on spectral elements too. From nek5000-users at lists.mcs.anl.gov Mon Oct 25 07:20:55 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Mon, 25 Oct 2010 14:20:55 +0200 Subject: [Nek5000-users] scaling of nek5000 on a Cray In-Reply-To: References: <1592648673.129041287557765924.JavaMail.root@zimbra.anl.gov> Message-ID: <4CC57627.3050306@mech.kth.se> Thanks Paul, Our system is currently down, but I'll do as you suggest as soon as the system is up and running again. Johan nek5000-users at lists.mcs.anl.gov wrote: > > Hi Johan, > > I've added some performance tests to the repo (v. 570) > > There are a couple of calling options - > > I would suggest, in usrdat, within your .usr file to have > > > iverbose = 1 ! fully verbose > call platform_timer(iverbose) > call exitti('timer done$',iverbose) > > and run this with a reasonably large 3D case on your parallel > platform. > > If you can send us the output of this run off list we'll > have a pretty good idea of what's going on. > > Thanks! > > Paul > > > > > On Wed, 20 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > >> >> I'm hoping to include our ping-pong and mxm timers in a future >> source release so that we can get a handle on the platform >> parameters via a couple of quick calls from userchk. With luck, >> we'll have this in a few days. -- Paul >> >> >> >> On Wed, 20 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: >> >>> Hi Johan, >> >> we can definitely do better on Cray systems but I guess we need to do >> some fine tuning first. >> The BG architecture is quite different and you need much more grid >> points per core to scale on the Cray (4-8x more). >> >> - Can you post your logfiles? >> - Did you try to enable Nek's internal MPI profiling where you spend >> most of the communication cost? >> - Did you try to use an optimized MPI rank mapping? >> >> Is it possible to get an account on that machine to do some experiments? >> >> >> Cheers, >> Stefan >> >> >> ----- Original Message ----- >> From: nek5000-users at lists.mcs.anl.gov >> To: nek5000-users at lists.mcs.anl.gov >> Sent: Tue, 19 Oct 2010 05:18:41 -0600 (GMT-06:00) >> Subject: [Nek5000-users] scaling of nek5000 on a Cray >> >> Hi nek5000-users & developers, >> >> I wonder if anyone has experience with the performance of nek5000 on a >> Cray ? In particular what scaling could one expect compared to e.g. a >> Blue Gene? >> >> We have a new Cray XT6m system, based on the AMD Opteron 12-core >> ?Magny-Cours? (2.1 >> GHz) processors and the Cray SeaStar2 interconnect technology. It >> consists of 11040 compute cores and 10 service cores. >> >> The scaling is not too encouraging (see attached pdf-files). The >> attached files show three different runs, and the scaling is apparently >> highly dependent on where in the torus the job happens to be. This is >> maybe ok, but is the leveling off after 2048 cores consistent with what >> people observe on a Cray ? >> >> Thanks for your suggestions and experience! >> >> Best regards, >> >> Johan >> >> -- >> Johan Malm >> Department of Mechanics, KTH >> SE-100 44, Stockholm, Sweden >> Phone: +46 8 7906876 >> E-mail: johan at mech.kth.se >> >> >> _______________________________________________ >> Nek5000-users mailing list >> Nek5000-users at lists.mcs.anl.gov >> https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > ------------------------------------------------------------------------ > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > -- Johan Malm Department of Mechanics, KTH SE-100 44, Stockholm, Sweden Phone: +46 8 7906876 E-mail: johan at mech.kth.se From nek5000-users at lists.mcs.anl.gov Fri Oct 29 12:56:54 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 29 Oct 2010 12:56:54 -0500 Subject: [Nek5000-users] Finding the maximum Message-ID: Hi, For a computation with eddy viscosity, I would like to find the maximum eddy viscosity within a particular region of my domain and also the location of the max/min value. Is there a routine in nek to give limits on x,y,z and then find max of a particular variable ? glmax() finds the maximum in the entire domain . . . Regards Shriram -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Oct 29 13:00:18 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 29 Oct 2010 11:00:18 -0700 Subject: [Nek5000-users] Finding the maximum In-Reply-To: References: Message-ID: Hi Shriram, If you are willing to constrain your search to the time slices you write out to disk, you should be able to do this fairly easily with VisIt. The first step would be isolating that domain. You would do that with the Box operator. Then you would apply the "Max" query. That will give you the maximum value and also the location. Make sure you click "Actual data" when you do the query ... it will restrict the search to the box. (Original data is for what's in the file.) Best, Hank On Fri, Oct 29, 2010 at 10:56 AM, wrote: > Hi, > > For a computation with eddy viscosity, I would like to find the maximum > eddy viscosity within a particular region of my domain and also the location > of the max/min value. > Is there a routine in nek to give limits on x,y,z and then find max of a > particular variable ? glmax() finds the maximum in the entire domain . . . > > Regards > Shriram > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Fri Oct 29 15:04:44 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 29 Oct 2010 15:04:44 -0500 (CDT) Subject: [Nek5000-users] Finding the maximum In-Reply-To: References: Message-ID: Hi Shriram, You can do this in userchk as follows: n = nx1*ny1*nz1*nelv vmax = -9e20 do i=1,n x=xm1(i,1,1,1) y=ym1(i,1,1,1) z=zm1(i,1,1,1) if (x0.le.x.and.x.le.x1 .and. $ y0.le.y.and.y.le.y1 .and. $ z0.le.z.and.z.le.z1 ) $ vmax=max(vmax,eddyv(i,1,1,1)) enddo vmax = glmax(vmax,1) where x0,x1,...,z1 are discrimators of your choice and eddyv, say, is the array in question. Paul On Fri, 29 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > Hi, > > For a computation with eddy viscosity, I would like to find the maximum eddy > viscosity within a particular region of my domain and also the location of > the max/min value. > Is there a routine in nek to give limits on x,y,z and then find max of a > particular variable ? glmax() finds the maximum in the entire domain . . . > > Regards > Shriram > From nek5000-users at lists.mcs.anl.gov Fri Oct 29 15:41:04 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Fri, 29 Oct 2010 15:41:04 -0500 Subject: [Nek5000-users] Finding the maximum In-Reply-To: References: Message-ID: Hi Hank and Paul , Both the method works. I am looking at time dependent max values of eddy viscosity and so have this routine by Paul implemented in nek. The one on VisIt helps me in getting the location of max values. Thanks ! Regards Shriram On 29 October 2010 15:04, wrote: > > Hi Shriram, > > You can do this in userchk as follows: > > > n = nx1*ny1*nz1*nelv > vmax = -9e20 > do i=1,n > x=xm1(i,1,1,1) > y=ym1(i,1,1,1) > z=zm1(i,1,1,1) > if (x0.le.x.and.x.le.x1 .and. > $ y0.le.y.and.y.le.y1 .and. > $ z0.le.z.and.z.le.z1 ) > $ vmax=max(vmax,eddyv(i,1,1,1)) > enddo > vmax = glmax(vmax,1) > > where x0,x1,...,z1 are discrimators of your choice > and eddyv, say, is the array in question. > > Paul > > > > > On Fri, 29 Oct 2010, nek5000-users at lists.mcs.anl.gov wrote: > > Hi, >> >> For a computation with eddy viscosity, I would like to find the maximum >> eddy >> viscosity within a particular region of my domain and also the location of >> the max/min value. >> Is there a routine in nek to give limits on x,y,z and then find max of a >> particular variable ? glmax() finds the maximum in the entire domain . . . >> >> Regards >> Shriram >> >> _______________________________________________ > Nek5000-users mailing list > Nek5000-users at lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nek5000-users at lists.mcs.anl.gov Sat Oct 30 09:37:31 2010 From: nek5000-users at lists.mcs.anl.gov (nek5000-users at lists.mcs.anl.gov) Date: Sat, 30 Oct 2010 09:37:31 -0500 Subject: [Nek5000-users] Grid 2 Grid Interpolation In-Reply-To: References: Message-ID: Neks, We are trying to get the grid to grid interpolation routine working. I am using a simple box as a test case, with a coarse grid of 64 as the "old geometry" that I've let go through several flow throughs, and a finer grid of 700 elements that we want to interpolate on to. I went through the steps on the website given for grid to grid interpolation, but on the last step when I run old geometry with g2gi called in usrchk, I get the following error message in the output file. call usrchk grid-to-grid interpolation ABORT: nelr>lelr on rank 0 ABORT: nelr>lelr on rank 1 call exitt: dying ... This occurred when I ran the old geometry on two processors. When I ran it on one processor, it gave the same message, except it didn't have the 2nd ABORT message. Any help would be much appreciated, as grid2grid should help us tremendously on the full case we are trying to get through. Thanks, Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: