Parallel Netcdf Feature

[Jianwei Li]

Yu-heng,

I read your test code.
There are two subarray settings you need:

1. since array is partitioned, each process will only write
   part of the whole array by calling our "put_vara" functions. 
   This is file subarray.
   
2. process 0 chooses not to output all its part, thus you need
   a memory subarray to select regions of local data to feed to 
   the I/O. This is achieved by using the MPI subarray derived 
   datatype for the memory data (buffer).
   
   e.g. MPI_Type_create_subarray(..., MPI_REAL, &mysubarray, ierr)
        nfmpi_put_vara_all(..., buffer, mysubarray)
        
   Note, "mysubarray" specifies to choose only second row instead
   of both rows on process 0. Here we introduce using our Flexible
   API which allows user to specify memory data layout (choosing
   regions of data buffer for output), instead of using traditional
   per-type function "nfmpi_put_vara_real_all()". The element type 
   and memory layout are incorporated in "mysubarray" as a MPI derived
   data type.

Your test code only did the first step, so file region was correctly
selected, but the memory data was not properly selected to feed into
the file region.

Let me know if it doesn't work.

Thanks!
   
Jianwei
   
>Jianwei,
>
>Thanks for your explanation.
>Do I really need to use MPI subarray datatype?
>I attempted to use subarray function put_vara_*
>It seems I couldn't get what I want.
>
>Do you have any test example to verify this function?
>I tested it using the test code in test/fandc/pnf_test.F
>
>For example, ARRAY has dimension (4,3), e.g.
>    ARRAY=[ 1, 2, 3,
>            5, 6, 7,
>            9,10,11,
>           13,14,15]
>So ARRAY(1,1)=1; ARRAY(1,2)=2; ARRAY(1,3)=3;
>   ARRAY(2,1)=5; ARRAY(2,2)=6;
>
>If I use two processes to output by using nfmpi_put_vara()
>setting start()=[1,1],count()=[3,2] for process 0
>Process 0 outputs
>1,2,3,
>5,6,7,             
>
>setting start()=[1,3],count()=[3,2] for process 1
>process 1 outputs
>9,10,11,
>13,14,15
>
>So I use "ncdump pnf_test.nc" and get
> 1, 2, 3,
> 5, 6, 7,
> 9,10,11,
>13,14,15
>This looks fine to output the whole array.
>However, when I modify start() and count() as you suggested
>to output only part of the subarray, I didn't get what I want.
>In process 0, I set start()=[1,2], count()=[3,1] (ignore 1st row)
>In process 1, I set start()=[1,3], count()=[3,2] (the same)
>
>It gives me
> 0, 0, 0,
> 1, 2, 3,
> 9,10,11,
>13,14,15
>
>It appears that the output always starts from
>the first element rather than the starting location I specify
>from the start(). Am I doing something wrong?
>My objective is to output 
> 5, 6, 7,
> 9,10,11,
>13,14,15
>(ignore 1st row)
>
>But I don't think it can work for parallel netCDF.
>It will be great if you can provide any example. Thanks!
>
>Yu-heng
>
>----- Original Message -----
>From: Jianwei Li <jianwei at ece.northwestern.edu>
>Date: Monday, June 20, 2005 8:53 am
>Subject: Re: Parallel Netcdf Feature
>
>> 
>> Yu-heng,
>> 
>> I did not quite catch what you want, but basically
>> our subarray functions put/get_vara can exclude
>> boundary columns/rows as many as you want to.
>> You just need to specify start[] and count[] vectors
>> for the subarray, e.g., process 0 can specify
>> start[] = {0, 1} and count[] = {3, 1} while process 1
>> can specify start[] = {0, 2} and count[] = {3, 2}.
>> 
>> Thanks,
>> Jianwei
>> 
>> On Sun, 19 Jun 2005, Yu-Heng Tseng wrote:
>> 
>> > Hi
>> >
>> > I am interested in doing some complicated output
>> > using parallel NETCDF so that each process outputs
>> > different sizes.
>> > For example, the total size has two dimensions (x,y).
>> > If the total size is (4,3), e.g.
>> >
>> > 1,2,3,4,
>> > 5,6,7,8
>> > 9,10,11,12
>> >
>> > If we use two processes:
>> > Process 0 has  and process 1 has
>> > 1,2,               3,4,
>> > 5,6,               7,8,
>> > 9,10               11,12
>> > I don't want to output the first column
>> > (which includes 1,5,9). How can I output
>> > the rest 9 variables
>> > 2,3,4,
>> > 6,7,8,
>> > 10,11,12
>> >
>> > Can I obtain some examples? or can someone
>> > tell me any particular subroutine I can use?
>> > I can check it out. There are not many test
>> > examples in the current release.
>> > Thank you very much.
>> >
>> > Yu-heng
>> >
>> 
>>