Unchecked memory allocation and potential performance problem

[William Gropp]

On the copy step, I wonder if a code that tried harder to work with  
aligned blocks and in the parallel case tried to avoid conflicts  
between processors would be valuable.  It might be an interesting  
class exercise: Given the needs here (which may be unaligned  
segments), design, implement, verify, and measure the performance of  
an implementation of the parallel move operation.  The winning code  
could become part of pnetCDF (and perhaps part of netCDF as well).

Are there other places in the pnetCDF code where it might be valuable  
to either explicitly align and block the I/O, or ensure that the MPI- 
IO call has enough to work with so that a high-quality MPI-IO  
implementation would do the same?

Bill

On Dec 5, 2006, at 3:06 PM, Robert Latham wrote:

>
>> Also, the 4096 is used
>> as the buffer size in MPI_File_read_at and similar calls (this seems
>> rather small).  Should this buffer size be negotiated? (I'm wondering
>> if this might be a source of the slowdown that we're seeing, as this
>> code can be invoked during a close).
>
> I covered most of this in my reply to Katie's message.  I don't know
> what a good value would be for this:  no matter how big we make it, if
> the caller mucks with the header and forces us to shuffle all the bits
> in the datafile, it's going to be slow.  Additionally, both we and  
> serial
> netcdf strongly encourage everyone to do all their defining once, so
> is it worth constructing a lot of negotiation infrastructure for a
> hopefully infrequenlty used code path?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/parallel-netcdf/attachments/20061206/39a54c6b/attachment.htm>