Paul,<br><div class="gmail_quote"><br>Thanks for the starting points. I have a few more questions, especially about the parallelization that's necessary for these huge systems. I don't really care about speed here, I just want to output a 2d slice and I don't plan on doing it very often.<br>
<br>1.) for z_slice_g(), what is ezi? Say I want to output all the data at z=0 (i.e. the bottom plate). What would I use for ezi? Or do I have to create some routine to find ezi for this case?<br><br>2.) I am a little confused about the parallel version of outfld2d(). It looks like outfld2d_p gets called by buff_2d_out(). Would you suggest doing this?<br>
<br>Thanks,<br><br>Janet<br><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><blockquote type="cite"><div><br>Hi Janet,<br><br>
I added z_slice_g, a generalized z_slice() extraction routine<br>to the repo in navier5.f.<br><br>I've not tested it.<br><br>Please let me know if this works for you. Note that all it<br>does is to extract a scalar. You would call this once for each<br>
variable of interest and then dump them as a field using some<br>type of output routine --- outfld2d() and outfld2d_p() look<br>like decent starting points but are not quite what you need.<br><br>If I were you, I would start with the binary .fld (not .f0000)<br>
format from a single processor and see how long that takes.<br><br>I think the _p() variant does the following ---<br><br>For several processors, it holds the data from different output<br>times (e.g., proc 0 holds time 1, proc 64 holds time 2, etc.).<br>
Then, at an agreed-upon timestep, each of these opens a file,<br>in parallel, and writes the entire z-slice that it holds as a copy.<br><br>Since each processor gets an entire z-slice from the above routine,<br>this is not a big deal, and you still get parallel I/O --- it's<br>
time parallel instead of space parallel. Here's a cartoon:<br><br><br>time/space: p0 p4 p12 p16<br><br>step 1000 get slice get slice get slice get slice<br>step 1000 save slice<br>
<br>step 2000 get slice get slice get slice get slice<br>step 2000 save slice<br><br>step 3000 get slice get slice get slice get slice<br>step 3000 save slice<br><br>
step 4000 get slice get slice get slice get slice<br>step 4000 save slice<br><br>step 4000 write f1 write f2 write f3 write f4 <-- Parallel IO<br><br>The attraction of this approach is that the write() looks<br>
like serial I/O because the requisite data is fully resident<br>on the processor that is doing the write, so it is simpler<br>to code.<div><div class="h5"><br><br>Paul<br><br>On Tue, 25 Jun 2013, <a href="mailto:nek5000-users@lists.mcs.anl.gov" target="_blank">nek5000-users@lists.mcs.anl.gov</a> wrote:<br>
<br><blockquote type="cite"><br></blockquote><blockquote type="cite">Hi Janet,<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I think that z_slice() and outfld2d would be reasonable<br></blockquote>
<blockquote type="cite">starting points.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">As I look at the code, z_slice takes element 1, slice 1<br></blockquote><blockquote type="cite">in the z direction. You would want to generalize this.<br>
</blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Also, it looks outfld2d() is set up only for ascii output<br></blockquote><blockquote type="cite">at the moment. I would suggest conversion to binary.<br>
</blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I can take a pass at making these changes but will not have<br></blockquote><blockquote type="cite">time to verify functionality for a few days. I could however<br>
</blockquote><blockquote type="cite">work with you on that.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Paul<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">
<br></blockquote><blockquote type="cite">On Tue, 25 Jun 2013, <a href="mailto:nek5000-users@lists.mcs.anl.gov" target="_blank">nek5000-users@lists.mcs.anl.gov</a> wrote:<br></blockquote><blockquote type="cite"><br></blockquote>
<blockquote type="cite"><blockquote type="cite">We are generating huge 3d data files (>8GB with multiple I/O) which are so<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">large, our version of visit does not have enough memory to load them.<br>
</blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Oftentimes we only want to visualize a 2d slice anyway. Is there a routine<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">
in nek5000 that does this? Or one that can be easily modified? To be more<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">specific, we want to output, for example, the coordinates and the velocity<br>
</blockquote></blockquote><blockquote type="cite"><blockquote type="cite">field for some fixed z-value, into a blah.f00001 file that can then be read<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">
by visit.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Thanks,<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Janet<br></blockquote></blockquote><blockquote type="cite">
<blockquote type="cite">-- <br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Janet Scheel<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Assistant Professor of Physics<br>
</blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Occidental College<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">1600 Campus Road, M21<br></blockquote></blockquote>
<blockquote type="cite"><blockquote type="cite">Los Angeles, CA 90041<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="tel:%28323%29%20259-2777" value="+13232592777" target="_blank">(323) 259-2777</a><br>
</blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="mailto:jscheel@oxy.edu" target="_blank">jscheel@oxy.edu</a><br></blockquote></blockquote><blockquote type="cite">_______________________________________________<br>
</blockquote><blockquote type="cite">Nek5000-users mailing list<br></blockquote><blockquote type="cite"><a href="mailto:Nek5000-users@lists.mcs.anl.gov" target="_blank">Nek5000-users@lists.mcs.anl.gov</a><br></blockquote>
<blockquote type="cite"><a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br></blockquote><blockquote type="cite"><br></blockquote>
_______________________________________________<br>Nek5000-users mailing list<br><a href="mailto:Nek5000-users@lists.mcs.anl.gov" target="_blank">Nek5000-users@lists.mcs.anl.gov</a><br><a href="https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users</a><br>
</div></div></div></blockquote></div><br></div></blockquote></div><br><br clear="all"><br>-- <br>Janet Scheel<br>Assistant Professor of Physics<br>Occidental College<br>1600 Campus Road, M21<br>Los Angeles, CA 90041<br>(323) 259-2777<br>
<a href="mailto:jscheel@oxy.edu">jscheel@oxy.edu</a>