non contiguous data IO

[Robert Latham]

On Mon, Oct 02, 2006 at 02:41:35PM +0200, Seiss, Guntram wrote:
> when using triangulated networks, we use an area splitting method to
> distribute the work on to sevaral processors.
> On file, it should be possible, to distribute the data non contiguos.
> As an example I wrote a test program using MPI-IO and an user defined
> fileview.
>  
> The fileviews are constructed as follows:
>              P0 |0xxxx0000xxxxxxxxxx00xxxx|  
>              P1 |x1111xxxx1111xxxxxxxxxxxx| 
>              P2 |xxxxxxxxxxxxx222222xxxxxx|
>              P3 |xxxxxxxxxxxxxxxxxxxxx3333|
>  
>           total |0111100001111222222003333|
> 
>  Now I would like to realize the same in pnetCDF. Does anyone know about
> examples, how I can use user defined
> MPI-datatypes to describe such pattern like the above one?

I think pnetcdf can help you out.  In our implementation, parallel
NetCDF creates file views much like you did by hand, but because
parallel-NetCDF creates NetCDF data files, there is a well-defined
structure to the generated file.  If you are ok with the datafile
format defined by NetCDF, then I think Parallel-NetCDF will do
something close to what you want, without a whole lot of effort on
your part.

With parallel-netcdf, you can described your data with pnetcdf
routines and then the write operation (ncmpi_put_vara_all, for
example) will carry out the I/O in parallel.  You will probably not
get the exact distribution of processes and I/O that you laid out
above, but you will have each process read/write its own region of the
netcdf datafile.  

Let us know if my answer is in any way confusing.  If you start
experimenting with parallel-netcdf, let us know how it goes.  we'll be
happy to help out if you run into problems.

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B