[MPICH] slow IOR when using fileview

[Wei-keng Liao]

Weikuan,

I found this problem when I ran IOR benchmark. I extracted a simpler code 
to reproduce this situation (it is provided in my earlier post on this 
list.) The 10MB is used by this simpler code in order to show the 
performance difference I mentioned. It does not mean I used only 10 MB in 
my IOR runs.

Wei-keng


On Mon, 2 Jul 2007, Yu, Weikuan wrote:

>
> Thanks for reporting this. I got a similar report from a ticket you
> filed at ORNL. I am following up with this thread for folks' collective
> attention on this behavior.
>
> While there are many differences between Cray XT and other platforms
> with a linux-based Lustre file system or PVFS, such as caching and comm
> library etc, this performance difference between write_all and
> write_at_all does not seem to be directly related to them. Besides,
> running collective IO with IOR may not seem to be an advisable thing
> with the intended pattern of 10MB per file? Could you detail a little
> more on the actual need/intention of a file view here, or a breif
> description of intended access pattern in your apps?
>
> --
> Weikuan Yu, Ph.D
> Future Technologies & Technology Integration
> Oak Ridge National Laboratory
> Oak Ridge, TN 37831-6173
> Email: wyu at ornl.gov
> http://ft.ornl.gov/~wyu/
>
>> -----Original Message-----
>> From: owner-mpich-discuss at mcs.anl.gov
>> [mailto:owner-mpich-discuss at mcs.anl.gov] On Behalf Of Wei-keng Liao
>> Sent: Saturday, June 30, 2007 3:03 AM
>> To: mpich-discuss at mcs.anl.gov
>> Subject: [MPICH] slow IOR when using fileview
>>
>>
>> I am experiencing slow IOR performance on Cray XT3 when using
>> fileview option. I extract the code into a simpler version
>> (attached). The code compares two collective writes:
>> MPI_File_write_all and MPI_File_write_at_all. The former uses
>> an MPI fileview and the latter uses explicit file offset. For
>> both cases, each process writes 10 MB to a shared file,
>> contiguously, non-overlapping, non-interleaved. On the Cray
>> XT3 with Lustre file system, the former is extremely slower
>> than the latter. Here is an output for using 8 processes:
>>
>> 2: MPI_File_write_all() time = 4.72 sec
>> 3: MPI_File_write_all() time = 4.74 sec
>> 6: MPI_File_write_all() time = 4.77 sec
>> 1: MPI_File_write_all() time = 4.79 sec
>> 7: MPI_File_write_all() time = 4.81 sec
>> 0: MPI_File_write_all() time = 4.83 sec
>> 5: MPI_File_write_all() time = 4.85 sec
>> 4: MPI_File_write_all() time = 4.89 sec
>> 2: MPI_File_write_at_all() time = 0.02 sec
>> 1: MPI_File_write_at_all() time = 0.02 sec
>> 3: MPI_File_write_at_all() time = 0.02 sec
>> 0: MPI_File_write_at_all() time = 0.02 sec
>> 6: MPI_File_write_at_all() time = 0.02 sec
>> 4: MPI_File_write_at_all() time = 0.02 sec
>> 7: MPI_File_write_at_all() time = 0.02 sec
>> 5: MPI_File_write_at_all() time = 0.02 sec
>>
>> I tried the same code on other machines and different file
>> systems (eg.
>> PVFS), and timings for both cases were very close to each
>> other. If anyone has access to a Cray XT3 machine, could you
>> please try it and let me know?
>> Thanks.
>>
>> Wei-keng
>>
>