[mpich-discuss] ROMIO hints start_iodevice, striping_unit, and striping_factor

[Wei-keng Liao]

On Apr 21, 2009, at 1:17 PM, Rob Latham wrote:
> On Tue, Apr 21, 2009 at 12:20:34PM -0500, Wei-keng Liao wrote:
>> I found the ROMIO hint start_iodevice appears in
>> ad_pvfs/ad_pvfs_open.c, but is not defined in include/adioi.h. So,
>> this hint is ignored by  ROMIO.  Can I suggest to add this to
>> ADIOI_Hints_struct?
>
> Hi Wei-keng.
>
> we can definitely add more things to ADIOI_Hints_struct.  Did you have
> a particular use in mind for it?
>
Hi, Rob,

I only have a rough idea but believe there is a potential to improve I/O
if this is available in ROMIO, users have more control for striping
configuration.


>> Two other hints, striping_unit and striping_factor, are defined but
>> currently are only set by taking user's inputs. None of the ADIO
>> drivers, except  PVFS-1, sets these hints by querying the file
>> system. It would be nice to see  PVFS-2, Lustre, and GPFS (BGL) with
>> this implemented, so users can query a  file's striping info through
>> MPI_File_get_info(). In addition, file domain  alignment depends on
>> the value striping_unit. It is important to detect this value
>> automatically.
>>
>> One approach to add this functionality is to query the stripe info
>> in ADIOI_xxx_Open by process fd->hints->ranklist[0] and broadcast it
>> to the rest of processes (including cb_nodes and non-cb_nodes). The
>> broadcast can be done in ADIO_Open() in ad_open.c
>
> We recently (3 weeks ago) did something like this for ROMIO on
> BlueGene.  I suppose the stat() interface is a lot more standard now
> than 10 years ago.  We can probably move this out of blueGene and into
> a generic form with little risk.
>

I know Lustre uses ioctl() to get/set the striping parameters.
I would suggest to keep this get/set mechanism private for individual
file systems (as different file system may use different system call
for this), but broadcast the striping hints in a generic form, i.e.
in ADIO_Open().

Wei-keng


> ==rob
>
> -- 
> Rob Latham
> Mathematics and Computer Science Division
> Argonne National Lab, IL USA
>