More on slow file output

Rob Ross rross at mcs.anl.gov
Wed Dec 6 10:52:39 CST 2006


Right, so if you create the attributes in the first define mode and then 
fill them in with accurate values later, all should be ok as long as you 
don't use larger strings in the second define mode.

And, it sounds like we're going to leave some space in the header so 
this isn't so expensive in the future.

Regards,

Rob

Robert Latham wrote:
> On Tue, Dec 05, 2006 at 02:48:13PM -0800, Katie Antypas wrote:
>> I think it is a pretty easy fix to just write out the max and min 
>> attributes initially in define mode, although it will require a number 
>> of changes to the FLASH IO internal interfaces.  I didn't realize redef 
>> would cause such a performance hit.  Right now I'm running the tests 
>> without writing the max and min attributes which I don't think are all 
>> that important in any case.
> 
> I wanted to make sure I clarified a point:  If in the second define
> mode phase you modify existing attributes without changing their size,
> then there won't be a need to rewrite the entire file.  
> 
> You'd have to be careful with string attributes for example, but
> changing a float atribute called "max_pressure" from -1.0 to 642.234
> won't cause a rewriting, since both values fit into the same 4 byte
> space allocated in the first define mode phase.
> 
> In short, entering define mode twice isn't necessarily expensive.
> It's expensive if you change the size of the information you defined
> the first time.
> 
>> The 128 proc case ran with no problems.  I'm waiting for some larger 
>> jobs to go through.  I'll let you guys know the results.
> 
> excellent! thanks
> ==rob
> 




More information about the parallel-netcdf mailing list