[mpich2-dev] improved scalability with O_NOATIME flag

[Rob Latham]

Came across an interesting optimization that should be easy to add to
ROMIO, but I want to do so in a portable way.

the O_NOATIME flag does what it says on the tin: don't update atime
when you open this file.  Now O_RDONLY really does only read -- no
metadata update needed.

In a collective open we can set this flag on all processors save one,
and presumably avoid a metadata storm (think lustre and it's single
metadata server).

So, what's the "MPICH way" to make use of a gnu-libc flag?  On my
laptop it's protected by an "#ifdef __USE_GNU".  Is it ok
to write a configure-time check for O_NOATIME that defines __USE_GNU?
But then I have to set __USE_GNU inside adio/common/ad_open.c if
we HAVE_O_NOATIME .

==rob

----- Forwarded message from Mark Howison <mark.howison at gmail.com> -----

Sender: hdf-forum-bounces at hdfgroup.org
From: Mark Howison <mark.howison at gmail.com>
Reply-To: HDF Users Discussion List <hdf-forum at hdfgroup.org>
Subject: Re: [Hdf-forum] round-robin (not parallel) access to single hdf5
	file
Date: Tue, 14 Dec 2010 12:03:05 -0500
Message-ID: <AANLkTin92tPWF1r3XhvaCxNB1sMpkQxtt0qx1FLaE==u at mail.gmail.com>
To: HDF Users Discussion List <hdf-forum at hdfgroup.org>
X-Spam-Status: No, score=-2.1

On Tue, Dec 14, 2010 at 8:08 AM, Quincey Koziol <koziol at hdfgroup.org> wrote:
>> If not, there is another optimization that I think was reported in a
>> paper on PLFS or Adios about passing a flag to the fopen call on each
>> MPI task that tells it not to update the creation/modification time
>> except on the root task. This can greatly reduce the load on the
>> metadata server for a parallel file system.
>
>        Interesting, can you send me a reference for this?

I'm pretty sure the trick was to use O_NOATIME in the open() call,
except on task 0. (You can find this on NICS webpage on I/O best
practices.)

I know I came across it in the lit review for our HDF5/Lustre paper,
but I can't put my fingers on the paper. It might haveI vaguely recall
a scaling graph showing how this outperformed a regular open() call,
and I think the text was 1-column wide... I'll keep looking through my
woefully unorganized pile of PDFs.

Mark

_______________________________________________
Hdf-forum is for HDF software users discussion.
Hdf-forum at hdfgroup.org
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org

----- End forwarded message -----

-- 
Rob Latham
Mathematics and Computer Science Division
Argonne National Lab, IL USA