pnetcdf-write-nb.c non-blocking

[Wei-keng Liao]

Another clarification, overlapped user buffers are not allowed
for nonblocking read operations. This is due to MPI-IO limitation
on prohibiting overlapping user buffer datatype in read operations.
See MPI 3.0 Chapter 13.4.1, the last sentence of page 506.

Wei-keng

On Sep 14, 2015, at 4:03 PM, Wei-keng Liao wrote:

> Hi, Rob,
> 
> You are right about the prohibition of reusing buffers in nonblocking APIs.
> 
> Just to make this limitation more clear. Reusing user buffers or using
> overlapped user buffers are prohibited in general. This limitation is not
> only for across nonlocking APIs but also for mixed nonblocking and blocking
> APIs. The reason is because the user buffer may be used internally by
> PnetCDF to do an in-place byte swap operation.
> 
> However, if PnetCDF is built with --disable-in-place-swap, then
> the write buffer will be copied to a temporary buffer before committed
> to the file. In this case, the write buffers might technically be reused
> or overlapped.
> 
> 
> Wei-keng
> 
> On Sep 14, 2015, at 2:45 PM, Rob Latham wrote:
> 
>> 
>> 
>> On 09/07/2015 04:25 PM, Wei-keng Liao wrote:
>>> Hi,
>>> 
>>> The example program is buggy.
>>> The write buffers for the two iput calls should be different.
>>> FYI. Once a nonblocking iput/iget API is posted, the contents
>>> of the buffers should not be touched before the call to
>>> ncmpi_wait_all/ncmpi_wait. Otherwise, the results can be incorrect.
>>> 
>>> Please see the bug fix in http://trac.mcs.anl.gov/projects/parallel-netcdf/changeset/2095
>>> Thanks for reporting this problem.
>> 
>> Hi Wei-keng:  I wrote that example and did indeed follow the rule that the contents of the buffers should not be touched before the call to ncmpi_wait_all. -- the 'data' buffer is the same for both calls and is unmodified.
>> 
>> My mistake was that I remembered the buffer reuse rule (for both MPI and pnetcdf) as "don't change the buffer" but it's more than that: I just wanted to make it clear that the rules for buffer reuse prohibit not only modification but any reads as well.
>> 
>> ==rob
>> 
>>> 
>>> Wei-keng
>>> 
>>> On Sep 7, 2015, at 1:52 PM, W. Miah wrote:
>>> 
>>>> Hello,
>>>> 
>>>> I have downloaded pNetCDF and am browsing through the example code
>>>> parallel-netcdf-1.6.0/examples/tutorial/pnetcdf-write-nb.c (attached
>>>> with this email) which should write the MPI ranks into two arrays, but
>>>> instead seems to be writing random values:
>>>> 
>>>> [miahw at bengal tutorial]$ ncdump output.nc
>>>> netcdf output {
>>>> dimensions:
>>>>   d1 = 4 ;
>>>> variables:
>>>>   int v1(d1) ;
>>>>   int v2(d1) ;
>>>> 
>>>> // global attributes:
>>>>       :string = "Hello World\n",
>>>>           "" ;
>>>> data:
>>>> 
>>>> v1 = 0, 16777216, 33554432, 50331648 ;
>>>> 
>>>> v2 = 0, 16777216, 33554432, 50331648 ;
>>>> }
>>>> 
>>>> I have looked at the code but can't see anything wrong with it. Could
>>>> it be that the library has a bug?
>>>> 
>>>> Regards,
>>>> 
>>>> --
>>>> web: http://miahw.wordpress.com
>>>> gnupg: 2E29 B22F
>>>> <pnetcdf-write-nb.c>
>>> 
>> 
>> -- 
>> Rob Latham
>> Mathematics and Computer Science Division
>> Argonne National Lab, IL USA
>