<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Thanks Russ (comments below)<br>
<br>
Russ Rew wrote (on 12/22/05 1:08 PM):
<blockquote cite="mid200512222108.jBML8j7s013609@unidata.ucar.edu"
type="cite">
<pre wrap="">Hi Dan,
</pre>
<blockquote type="cite">
<pre wrap="">I have a parallel code/machine which uses the standard netCDF library to
write data to a single file system NFS-mounted on the compute codes. As
I got this code, it uses a fairly simple MPI send/recv mechanism
(similar to token passing) to allow output files to be written by
multiple processors in a serialized fashion (each opens, writes, then
closes the file). This is slow, and bad!
As I don't know the netCDF library well, my first thought is simply to
have all the processors write at once, since they are going to write to
separate parts of the file. However, depending on how the library does
buffering, this may not work. (I know it wouldn't work with the cfitsio
package, for example.) Does anyone know if this will work, will not
work, or will have uncertain behavior?
</pre>
</blockquote>
<pre wrap=""><!---->
I know this will not work. The way the netCDF library buffers its
output to lessen the number of disk writes, multiple writers will in
general write over each other's disk blocks, even if they each write
separate areas of the output file.
</pre>
<blockquote type="cite">
<pre wrap="">Assuming that this will not always work, is parallel netCDF the next
choice, even when the file system is question is not a parallel file system?
</pre>
</blockquote>
<pre wrap=""><!---->
I think you need a parallel file system to do what you have in mind.
</pre>
</blockquote>
This is not an option, unfortunately. The 4 options are:<br>
1. Keep doing what I am doing, even though it is slow<br>
2. Accumulate data to node 0 in slabs, then have node 0 write out the
output file<br>
3. Figure out a way to do this in parallel using the standard <span>netcdf</span>
library that's faster than the current state.<br>
4. Use the parallel <span>netcdf</span> library, assuming that I can
use it without a parallel file system.<br>
<br>
Any other suggestions are welcome.<br>
<br>
Dan<br>
<blockquote cite="mid200512222108.jBML8j7s013609@unidata.ucar.edu"
type="cite">
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">Any other thoughts or suggestions?
</pre>
</blockquote>
<pre wrap=""><!---->
Some users have each process write a separate file, and subsequently
merge the many small files into one large netCDF file in a
post-processing step, but this can also be costly.
--Russ
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel S. Katz <a class="moz-txt-link-abbreviated" href="mailto:Daniel.S.Katz@jpl.nasa.gov">Daniel.S.Katz@jpl.nasa.gov</a>
Jet Propulsion Laboratory or <a class="moz-txt-link-abbreviated" href="mailto:d.katz@ieee.org">d.katz@ieee.org</a>
California Institute of Technology or <a class="moz-txt-link-abbreviated" href="mailto:dsk@computer.org">dsk@computer.org</a>
Mail Stop 126-104 (818) 354-7359 (voice)
4800 Oak Grove Drive (818) 393-6141 (fax)
Pasadena, CA 91109-8099 <a class="moz-txt-link-freetext" href="http://pat.jpl.nasa.gov/public/dsk/">http://pat.jpl.nasa.gov/public/dsk/</a></pre>
</body>
</html>