[mpich2-dev] More ROMIO performance questions

[Bob Cernohous]

We have another i/o scenario with interesting performance issues.

One again, it's large non-interleaved contiguous blocks being written/read 
(checkpointing software).  We ran into the same problems with data sieving 
and romio_cb_write/read = enable as we discussed a couple weeks ago.

We tried to tune it with hints for cb_block_size and get ok performance 
when we can avoid read/write data sieving.

Trying romio_cb_write/read = automatic gets very poor 
performance.Similarly, pure non-collective writes get very poor 
performance.  It seems like having too many writers/readers performs 
poorly on their configuration ... so

They customized the testcase to coordinate/flow-control the non-collective 
i/o and they get great performance.   They only have N simultaneous 
writers/readers active.  They pass a token around and take turns.  It's 
almost like having N aggregators but without the collective i/o overhead 
to pass the data around.  Instead they pass a small token and take turns 
writing the large, non-interleaved contiguous data blocks.

I'm not aware of anything in MPIIO or ROMIO that would do tihs?   Has this 
been explored by the experts (meaning you guys)? 



Bob Cernohous:  (T/L 553) 507-253-6093

BobC at us.ibm.com
IBM Rochester, Building 030-2(C335), Department 61L
3605 Hwy 52 North, Rochester,  MN 55901-7829

> Chaos reigns within.
> Reflect, repent, and reboot.
> Order shall return.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/mpich2-dev/attachments/20090914/057dcf2e/attachment.htm>