More on slow file output

Robert Latham robl at mcs.anl.gov
Tue Dec 5 17:19:50 CST 2006


On Tue, Dec 05, 2006 at 02:48:13PM -0800, Katie Antypas wrote:
> I think it is a pretty easy fix to just write out the max and min 
> attributes initially in define mode, although it will require a number 
> of changes to the FLASH IO internal interfaces.  I didn't realize redef 
> would cause such a performance hit.  Right now I'm running the tests 
> without writing the max and min attributes which I don't think are all 
> that important in any case.

I wanted to make sure I clarified a point:  If in the second define
mode phase you modify existing attributes without changing their size,
then there won't be a need to rewrite the entire file.  

You'd have to be careful with string attributes for example, but
changing a float atribute called "max_pressure" from -1.0 to 642.234
won't cause a rewriting, since both values fit into the same 4 byte
space allocated in the first define mode phase.

In short, entering define mode twice isn't necessarily expensive.
It's expensive if you change the size of the information you defined
the first time.

> The 128 proc case ran with no problems.  I'm waiting for some larger 
> jobs to go through.  I'll let you guys know the results.

excellent! thanks
==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Lab, IL USA                 B29D F333 664A 4280 315B




More information about the parallel-netcdf mailing list