[mpich-discuss] [wc-discuss] Re: testing new ADIO Lustre code

[Andreas Dilger]

On 2012-05-23, at 1:29 PM, Martin Pokorny wrote:
> Rob Latham wrote:
>> On Thu, May 17, 2012 at 03:02:48PM -0600, Martin Pokorny wrote:
>>> In broad terms, the changes I made are on two fronts: changing the
>>> file domain partitioning algorithm, and introducing non-blocking
>>> operations at several points. 
>> 
>> Non-blocking communication or i/o ?
>> One concern with non-blocking I/O in this path is that often the
>> communication and I/O networks are the same thing (e.g. infiniband, or
>> the BlueGene tree network in some situations).  
> 
> Non-blocking in both communication and I/O. I was preparing a question to the Lustre discussion list about non-blocking I/O using the POSIX aio API. I'll just ask right here, then. Is POSIX aio on a Lustre file system truly asynchronous? I expect that perhaps the implementation of aio in glibc may be asynchronous w.r.t. the calling thread, but I also wonder whether system calls to the Lustre client are asynchronous or not. Can anyone help me understand?
> 
> I have a little data suggesting that the aio calls do improve performance a bit, but this is a tentative conclusion.

I don't think the Lustre AIO calls are really async.  This isn't an area that has received much usage or attention, so I suspect the calls may only be wired up but will still block internally.

It might be that the 20 layers of IO submission API abstraction in the kernel avoids this problem for us, but I haven't looked at that recently.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/