[Nek5000-users] MPI-IO in Nek

[nek5000-users at lists.mcs.anl.gov]

Hi Hesam,

Regarding the 8192 tasks case, did you reduce your memory foot print
such that 8192*lelt ~ nel, the number of elements in your case?

I'm pretty certain that you just set p65=1, p66=p67=6 in usrdat2()
and all should work if you've compiled from scratch with the MPI-IO
flag.

In principle, the code will figure out from the header that your
input has multiple files, regardless of the value of p65, as should
be the case since the files dictate the input format, not some other
flag.

In practice, I know that the preceding statement is true for the
non MPI-IO case, but I'm not 100% certain for the MPI-IO case, which
is focussed more on the single-file approach.  I think however that
it works in this case as well.   If not, let me know --- there are
possible work-arounds.

Paul


----- Original Message -----
From: nek5000-users at lists.mcs.anl.gov
To: nek5000-users at lists.mcs.anl.gov
Sent: Saturday, October 19, 2013 12:49:37 PM
Subject: Re: [Nek5000-users] MPI-IO in Nek





Also, I asked this specifically because my largest job using 8192 tasks was killed at the first IO-time due to being out of memory although I was using the .f000 format (either when every processor was outputting or 64 of them) . Any ideas as to why that might have happened? 



Hesam 





From: nek5000-users-bounces at lists.mcs.anl.gov [mailto:nek5000-users-bounces at lists.mcs.anl.gov] On Behalf Of nek5000-users at lists.mcs.anl.gov 
Sent: Saturday, October 19, 2013 12:18 PM 
To: nek5000-users at lists.mcs.anl.gov 
Subject: [Nek5000-users] MPI-IO in Nek 



Hi Neks 



After reading the following post, the inherent differences b/w .fld and .f000 files became somehow clear to me. However, I have a few questions: 



https://lists.mcs.anl.gov/mailman/htdig/nek5000-users/2013-June/002147.html 



1) Is this correct that compiling with MPI-IO only requires adding this flag in makenek: PPLIST="MPIIO" ? or is there anything else I need to do? 

2) I noticed that if you do not add the above flag but set param(66) and param(67) in usrdat2 to 6 or -6 , it does generate .f000 files. I assume it implicitly used MPIIO, right? 

3) If I use several directories for outputting multiple files (by say setting p65=-64 in the .rea file), how can I restart from those multiple files associated with a single checkpoint in the * .rea * file? 



Thanks a lot for your help 



Best, 

Hesam 




_______________________________________________
Nek5000-users mailing list
Nek5000-users at lists.mcs.anl.gov
https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users