performance issue

[Wei-Keng Liao]

Is that the reason for why cb_nodes is 1?
Strange, because cb_nodes is set at the file open time.

Entering the independent data mode in PnetCDF can be completely avoided
if using the nonblocking APIs.

I would suggest your codes to use the nonblocking APIs in the following way.

/* for non-partitioned variables */
if (rank == 0) {
    ncmpi_iput_var_int(fh, varid[0], data[0], &req[0]); /* write the whole variable */
    ncmpi_iput_var_int(fh, varid[1], data[1], &req[1]);
    ...
}
/* for partitioned variables */
ncmpi_iput_vara_int(fh, varid[j], data[j], starts[j], counts[j], &req[j]);
...


/* commit all posted nonblocking requests */
ncmpi_wait_all(ncid, NC_REQ_ALL, NC_REQ_NULL, NULL);


Wei-keng

> On Aug 7, 2023, at 2:12 PM, Jim Edwards <jedwards at ucar.edu> wrote:
> 
> Hi Wei-Keng,
> 
> I think that I've found the problem.   In the model I am writing a number of scalar variables to the file as well as the decomposed variables.
> for the scalar variables I use a code structure like:
> 
> ncmpi_begin_indep_data(fh);
> ncmpi_put_vars_int(fh, varid, start, count, stride, data);
> ncmpi_end_indep_data(fh);
> 
> In my pioperf test code I didn't write any scalars - this morning I added one and the write performance for the decomposed variables got very very 
> bad.  What can I do about it?
> 
> Jim
> 
> 
> -- 
> Jim Edwards
> 
> CESM Software Engineer
> National Center for Atmospheric Research
> Boulder, CO