[Fwd: Re: [Swift-devel] Re: swift-falkon problem... plots to explain plateaus...]

[Michael Wilde]

We're only working on the BG/P system, and GPFS is the only shared 
filesystem there.

GPFS access, however, remains a big scalabiity issue. Frequent small 
accesses to GPFS in our measurements really slow down the workflow. We 
did a lot of micro-benchmark tests.

Zhao, can you gather a set of these tests into a small suite and post 
numbers so the Swift developers can get an understanding of the system's 
GPFS access performance?

Also note: the only local filesystem is RAM disk on /tmp or /dev/shm. 
(Ioan and Zhao should confirm if they verified that /tmp is on RAM).

- Mike

On 4/1/08 5:05 AM, Ben Clifford wrote:
> On Tue, 1 Apr 2008, Ben Clifford wrote:
> 
>>> With this fixed, the total time in wrapper.sh including the app is now about
>>> 15 seconds, with 3 being in the app-wrapper itself. The time seems about
>>> evenly spread over the several wrapper.sh operations, which is not surprising
>>> when 500 wrappers hit NFS all at once.
>> Does this machine have a higher (/different) performance shared file 
>> system such as PVFS or GPFS? We spent some time in november layout out the 
>> filesystem to be sympathetic to GPFS to help avoid bottlenecks like you 
>> are seeing here. It would be kinda sad if either it isn't available or you 
>> aren't using it even though it is available.
> 
>>From what I can tell from the web, PVFS and/or GPFS are available on all 
> of the Argonne Blue Gene machines. Is this true? I don't want to provide 
> more scalability support for NFS-on-bluegene if it is.
>