[MPICH] slow IOR when using fileview

[Yu, Weikuan]

Thanks for reporting this. I got a similar report from a ticket you
filed at ORNL. I am following up with this thread for folks' collective
attention on this behavior.

While there are many differences between Cray XT and other platforms
with a linux-based Lustre file system or PVFS, such as caching and comm
library etc, this performance difference between write_all and
write_at_all does not seem to be directly related to them. Besides,
running collective IO with IOR may not seem to be an advisable thing
with the intended pattern of 10MB per file? Could you detail a little
more on the actual need/intention of a file view here, or a breif
description of intended access pattern in your apps?

--
Weikuan Yu, Ph.D
Future Technologies & Technology Integration 
Oak Ridge National Laboratory
Oak Ridge, TN 37831-6173
Email: wyu at ornl.gov
http://ft.ornl.gov/~wyu/

> -----Original Message-----
> From: owner-mpich-discuss at mcs.anl.gov 
> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Wei-keng Liao
> Sent: Saturday, June 30, 2007 3:03 AM
> To: mpich-discuss at mcs.anl.gov
> Subject: [MPICH] slow IOR when using fileview
> 
> 
> I am experiencing slow IOR performance on Cray XT3 when using 
> fileview option. I extract the code into a simpler version 
> (attached). The code compares two collective writes: 
> MPI_File_write_all and MPI_File_write_at_all. The former uses 
> an MPI fileview and the latter uses explicit file offset. For 
> both cases, each process writes 10 MB to a shared file, 
> contiguously, non-overlapping, non-interleaved. On the Cray
> XT3 with Lustre file system, the former is extremely slower 
> than the latter. Here is an output for using 8 processes:
> 
> 2: MPI_File_write_all() time = 4.72 sec
> 3: MPI_File_write_all() time = 4.74 sec
> 6: MPI_File_write_all() time = 4.77 sec
> 1: MPI_File_write_all() time = 4.79 sec
> 7: MPI_File_write_all() time = 4.81 sec
> 0: MPI_File_write_all() time = 4.83 sec
> 5: MPI_File_write_all() time = 4.85 sec
> 4: MPI_File_write_all() time = 4.89 sec
> 2: MPI_File_write_at_all() time = 0.02 sec
> 1: MPI_File_write_at_all() time = 0.02 sec
> 3: MPI_File_write_at_all() time = 0.02 sec
> 0: MPI_File_write_at_all() time = 0.02 sec
> 6: MPI_File_write_at_all() time = 0.02 sec
> 4: MPI_File_write_at_all() time = 0.02 sec
> 7: MPI_File_write_at_all() time = 0.02 sec
> 5: MPI_File_write_at_all() time = 0.02 sec
> 
> I tried the same code on other machines and different file 
> systems (eg. 
> PVFS), and timings for both cases were very close to each 
> other. If anyone has access to a Cray XT3 machine, could you 
> please try it and let me know? 
> Thanks.
> 
> Wei-keng
>