More on slow file output

[Katie Antypas]

I'm am doing some io benchmarking with the FLASH code on the NERSC 
machine Bassi, (IBM Power 5) and I was also seeing some horrible 
performance results.  I wanted to investigate this more before writing 
the list, but it looks to me that that the slow down is caused when the 
FLASH IO tries to 'reenter' define mode with the ncmpi_redef call.

(Anshu, FLASH defines all the pnetcdf dimensions and variables ahead of 
time, then leaves define mode and writes the data.  However, we added a 
some functionality early in the summer to write out the maximum and 
minimum values of each unk variable and these are written out as 
attributes of the given variable.  We reenter define mode in the routine 
io_ncmpi_write_unknowns and io_ncmpi_write_unknowns_sp for plotfiles in 
order to write out the attributes.)

Should the FLASH io should be doing something differently?  Should we 
expect a performance hit from ncmpi_redef?  When I remove the calls to 
reentering define mode and do not write out the attributes the 
performance looks pretty good and in fact (on initial tests) appears to 
be faster than the equivalent hdf5 tests.  I still have a few jobs 
waiting in the queue here but as I get the results I will let you know.

Katie






William Gropp wrote:
> It looks like the problem may be in the close step.  Some part of this 
> appears to be causing GPFS a great deal of trouble, and the trouble 
> increases as the number of processors increases.  Any ideas?  Is there 
> an easy way to turn on performance instrumentation?
>
> Bill