Hints on improving performance with WRF and Pnetcdf

[Gerry Creager]

Rob Latham wrote:
> On Thu, Sep 02, 2010 at 06:23:42PM -0600, Craig Tierney wrote:
>> I did try setting the hints myself by changing the code, and performance
>> still stinks (or is no faster).  I was just looking for a way to not
>> have to modify WRF, or more importantly have every user modify WRF.
> 
> What's going slowly?  
> 
> If wrf is slowly writing record variables, you might want to try
> disabling collective I/O or carefully selecting the intermediate
> buffer to be as big as one record.
> 
> That's the first place I'd look for bad performance.  

Ah, but I'm seeing the same thing on Ranger (UTexas). I'm likely going 
to have to modify the WRF pnetcdf code to identify a sufficiently large 
stripe count (Lustre file system) to see any sort of real improvement.

More to the point, I see worse performance than with normal Lustre and 
regular netcdf. AND, there's no way to set MPI-IO-HINTS in the SGE as 
configured on Ranger. We've tried and their systems folk concur, so it's 
not just me saying it.

I will look at setting the hints file up but I don't think that's going 
to give me the equivalent of 64 stripe counts, which looks like the 
sweet spot for the domain I'm testing on.

Craig, one I have time to get back on to this, I think we can convince 
NCAR to add this as a bug release. I also anticipate the tweak will be 
on the order of 4-5 lines.

-- 
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.862.3983
Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843