[Darshan-users] [Ext] Re: Issues with Fio benchmark tool being profiled with Darshan

[Neeraj Rajesh]

Thank you Shane. I really appreciate that.

Thank you, stay safe
Yours sincerely
Neeraj Rajesh

Jul 22, 2020 08:35:15 Snyder, Shane <ssnyder at mcs.anl.gov>:

> Thanks for the update, Neeraj, and thanks for taking the time to test that out for us. That's unfortunate that the problem wasn't as simple as we hoped, but let me take an action item to do some testing with FIO to see if I can reproduce and find a workaround for the problem.
> 
> I'll let you know what I find.
> 
> --Shane
> 
> 
> ----------------------------------------
> From: Neeraj Rajesh <nrajesh at hawk.iit.edu>
> Sent: Tuesday, July 21, 2020 12:47 PM
> To: Snyder, Shane <ssnyder at mcs.anl.gov>
> Cc: darshan-users at lists.mcs.anl.gov <darshan-users at lists.mcs.anl.gov>
> Subject: Re: [Ext] Re: [Darshan-users] Issues with Fio benchmark tool being profiled with Darshan
> 
> Hello Shane
> 
> Sorry for the delay, it looks like there is still an issue with profiling Fio with darshan-master and the counters are still 0.
> 
> Thank you, stay safe
> 
> Yours sincerely
> 
> Neeraj Rajesh
> 
> On Wed, Jul 15, 2020 at 1:43 PM Neeraj Rajesh <nrajesh at hawk.iit.edu> wrote:
> 
>> Hello Shane
>> 
>> I am using the release version on the Darshan website, (3.2.1). I shall check out the master branch to see if I can generate accurate profiles and can report back on it.
>> 
>> Thank you, Stay safe
>> 
>> Yours Sincerely
>> 
>> Neeraj Rajesh
>> 
>> On Wed, Jul 15, 2020 at 1:35 PM Snyder, Shane <ssnyder at mcs.anl.gov> wrote:
>> 
>>> Hi Neeraj,
>>> 
>>> It looks like FIO might be using the 'openat()' call to open the files it is doing I/O to, which we only recently added support for. Assuming you aren't already, would it be possible for you to try the master branch of Darshan to see whether you can generate accurate profiles using that?
>>> 
>>> Thanks,
>>> 
>>> --Shane
>>> 
>>> 
>>> ----------------------------------------
>>> From: Darshan-users <darshan-users-bounces at lists.mcs.anl.gov> on behalf of Neeraj Rajesh <nrajesh at hawk.iit.edu>
>>> Sent: Tuesday, July 14, 2020 3:15 PM
>>> To: darshan-users at lists.mcs.anl.gov <darshan-users at lists.mcs.anl.gov>
>>> Subject: [Darshan-users] Issues with Fio benchmark tool being profiled with Darshan
>>> 
>>> Hello
>>> 
>>> I am trying to profile an Fio command, where I do roughly a gigabyte of reads or writes or mixed at a fixed block size.
>>> 
>>> eg.
>>> 
>>> ```
>>> 
>>> fio --filename=./test --size=1024MB --direct=1 --rw=randrw --bs=4k --ioengine=sync --iodepth=256 --numjobs=1 --name=test-job --minimal --stats=0
>>> 
>>> ```
>>> 
>>> I have tested it with the following 'ioengines':
>>> 
>>> - sync
>>> 
>>> - psync
>>> 
>>> - posixaio
>>> 
>>> - pvsync
>>> 
>>> I do get a darshan file, but all of the POSIX counters are 0, except for POSIX_F_META_TIME, POSIX_STATS, POSIX_FILE_ALIGNMENT and POSIX_MMAPS. I have ensured the operation is not simultaneously doing both POSIX and STDIO.
>>> 
>>> As per my configuration it should not be getting 0s especially for the counters POSIX_SIZE_WRITE_1K_10K, POSIX_WRITES, POSIX_READS, POSIX_BYTES_WRITTEN, POSIX_BYTES_READ etc.
>>> 
>>> Kindly advise what to do next.
>>> 
>>> Thank you for your time, Stay safe
>>> 
>>> Yours Sincerely
>>> 
>>> Neeraj Rajesh
>>> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/darshan-users/attachments/20200722/455f6c2c/attachment.html>