[MPICH] slow IOR when using fileview

[Wei-keng Liao]

My test code only generates a single, contiguous write in each process (no 
interleaving, no overlapping across any processes). If the independent 
access is used instead, I don't know why each write is divided into 512 KB 
chunks and locking is ever needed to guaranteed the atomic access of the 
10 MB contiguous file range. For this particular access pattern, ROMIO 
should not do read-modify-write at all.

BTW, I also see the performance improvement when disabling romio_ds_write 
hint. This applies to all sorts of type constructors.

Wei-keng



On Mon, 2 Jul 2007, Robert Latham wrote:

> On Mon, Jul 02, 2007 at 04:34:49PM -0500, Rajeev Thakur wrote:
>> But I don't think the lock functions get called in the collective I/O case.
>
> No, in this case the collective I/O case is falling back to the
> independent methods (and thus calling the lock functions).
>
> Let's take the collective write case.  Assuming cb_write has not been
> set to 'disabled', ROMIO will check for interleaving.  In Wei-keng's
> case, is there any interleaving? if not, interleave_count will be 0
> ADIOI_GEN_WriteStridedColl will fall back to independent accesses.
>
> For further confirmation you can run the test with testfs:
>
> [3/8] ADIOI_TESTFS_SetInfo called on foo.view
> [3/8]    calling ADIOI_GEN_SetInfo
> [3/8] ADIOI_TESTFS_Open called on foo.view
> [3/8] ADIOI_TESTFS_SetInfo called on foo.view
> [3/8]    calling ADIOI_GEN_SetInfo
> [3/8] ADIOI_TESTFS_WriteStridedColl called on foo.view
> [3/8]    calling ADIOI_GEN_WriteStridedColl
> [3/8] ADIOI_TESTFS_WriteStrided called on foo.view
> [3/8]    calling ADIOI_GEN_WriteStrided
> [3/8] ADIOI_TESTFS_ReadContig called on foo.view
> [3/8]    reading (buf = 0x2b37e3206010, loc = 3145728, sz = 524288)
> [3/8] ADIOI_TESTFS_WriteContig called on foo.view
> [3/8]    writing (buf = 0x2b37e3206010, loc = 3145728, sz = 524288)
> [3/8] ADIOI_TESTFS_ReadContig called on foo.view
> [3/8]    reading (buf = 0x2b37e3206010, loc = 3670016, sz = 524288)
> [3/8] ADIOI_TESTFS_WriteContig called on foo.view
> [3/8]    writing (buf = 0x2b37e3206010, loc = 3670016, sz = 524288)
> [3/8] ADIOI_TESTFS_Close called on foo.view
> [3/8] ADIOI_TESTFS_SetInfo called on foo.noview
> [3/8]    calling ADIOI_GEN_SetInfo
> [3/8] ADIOI_TESTFS_Open called on foo.noview
> [3/8] ADIOI_TESTFS_WriteStridedColl called on foo.noview
> [3/8]    calling ADIOI_GEN_WriteStridedColl
> [3/8] ADIOI_TESTFS_WriteContig called on foo.noview
> [3/8]    writing (buf = 0x7fffc77c1b00, loc = 3145728, sz = 1048576)
> [3/8] ADIOI_TESTFS_Close called on foo.noview
>
>
> ==rob
>
> -- 
> Rob Latham
> Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
> Argonne National Lab, IL USA                 B29D F333 664A 4280 315B
>