[MPICH] slow IOR when using fileview

[Yu, Weikuan]

>From what's available in the Argonne release of MPI-IO code, it appears that the execution flow goes to non-collective mode when this pattern of non-overlapping IO is detected. 

Then, as RobL pointed out, forcing ADIOI_GEN_ReadStrided_naive seems to avoid fcntl() altogether. I have verified this is also the case with Cray MPI-IO. With romio_ds_{read,write} disabled, the performance of write_all() is improved to the level that is comparable to write_at_all(). 

Cheers,
--
Weikuan


-----Original Message-----
From: Rajeev Thakur [mailto:thakur at mcs.anl.gov]
Sent: Mon 7/2/2007 5:34 PM
To: 'Robert Latham'; 'Wei-keng Liao'
Cc: Yu, Weikuan; mpich-discuss at mcs.anl.gov; Canon, Richard Shane; Hodson, Stephen W.; Renaud, William A.; Vetter, Jeffrey S.
Subject: RE: [MPICH] slow IOR when using fileview
 
But I don't think the lock functions get called in the collective I/O case.

Rajeev 

> -----Original Message-----
> From: owner-mpich-discuss at mcs.anl.gov 
> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Robert Latham
> Sent: Monday, July 02, 2007 4:14 PM
> To: Wei-keng Liao
> Cc: Yu, Weikuan; mpich-discuss at mcs.anl.gov; Canon, Richard 
> Shane; Hodson, Stephen W.; Renaud, William A.; Vetter, Jeffrey S.
> Subject: Re: [MPICH] slow IOR when using fileview
> 
> On Mon, Jul 02, 2007 at 03:41:56PM -0500, Wei-keng Liao wrote:
> > Can you let me know where in ROMIO this fcntl() is called? 
> I wonder why it 
> > is called when I used a fileview and not when fileview is not used.
> 
> Hi Wei-keng and others:
> 
> ROMIO calls fcntl to claim/release locks in the routine
> ADIOI_Set_lock.  ADIOI_Set_lock is the main function for
> the ADIOI_WRITE_LOCK, ADIOI_READ_LOCK, and ADIOI_UNLOCK macros.
> 
> Why are these lock/unlock calls being made when you use a fileview?
> As Rajeev said yesterday, you are passing in a type that is
> logically contiguous, but has a type signature such that ROMIO treats
> it as a noncontiguous type.   
> 
> When you don't use a file type, ROMIO treats the operation as
> contiuous and makes no fcntl calls.
> 
> If you make ROMIO use ADIOI_GEN_ReadStrided_naive somehow, would that
> get better performance?  Looks like you can get that by disabling data
> sieving.  
> 
> Rajeev already suggested increasing cb_buffer_size.  What if you also
> increased ind_wr_buffer_size and ind_rd_buffer_size?  
> 
> ==rob
> 
> -- 
> Rob Latham
> Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
> Argonne National Lab, IL USA                 B29D F333 664A 4280 315B
> 
>