Unchecked memory allocation and potential performance problem

[Robert Latham]

On Wed, Dec 06, 2006 at 02:55:04AM -0600, William Gropp wrote:
> On the copy step, I wonder if a code that tried harder to work with  
> aligned blocks and in the parallel case tried to avoid conflicts  
> between processors would be valuable.  It might be an interesting  
> class exercise: Given the needs here (which may be unaligned  
> segments), design, implement, verify, and measure the performance of  
> an implementation of the parallel move operation.  The winning code  
> could become part of pnetCDF (and perhaps part of netCDF as well).
> 
> Are there other places in the pnetCDF code where it might be valuable  
> to either explicitly align and block the I/O, or ensure that the MPI- 
> IO call has enough to work with so that a high-quality MPI-IO  
> implementation would do the same?

Off the top of my head there are two not-too-hard ways we can do this:

There's nothing in the CDF-1 or CDF-2 file format spec that prevents
us from using an arbitrarily large header to describe the data.  If we
know the right parameters for alignment and blocksize, we can pad the
header out to a useful point (which might somewhat reduce the chance a
re-definition would trigger a costly data shuffle).

Same thing for variables.  We don't *have* to place variables butting
up against each other.  They could also be padded out to beneficial
points in the file.  This change would be more invasive than padding
the header.

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Lab, IL USA                 B29D F333 664A 4280 315B