[Darshan-users] Instrumenting statically-linked applications

Wed Nov 28 18:20:46 CST 2012

Hi Phil,
So, v2.2.4-pre6 works on both Ranger and Lonestar. I can confirm that 
darshan generates a log file. However, I am a bit confused because when 
I parse the log file it says my application did not open any file and in 
fact I am at least generating several output files. Maybe I have to 
configure something or supply some flags at compilation time so I can 
capture the full I/O load generated by my application.

Do you think I should use the darshan-test application and see what the 
output looks like?

You are right one cannot build 100% static executables on Ranger and 
Lonestar. However, the dynamic libs such as libverbs are installed on 
each compute node so, it is not an issue. What is more important is that 
the darshan lib and all my other libs are statically linked because they 
are not deployed system wide. In any case, I am thinking I should have 
seen in the darshan log files that some I/O activity occurred because of 
these dynamic libs.

Thanks,
Dragos

Dragos Constantin, PhD

Research Associate
Department of Radiology
Stanford University
Lucas MRS Center
1201 Welch Rd., PS-055
Stanford CA 94305

Office: (650) 736-9961
Fax: (650) 723-5795

On 11/28/2012 11:25 AM, Phil Carns wrote:
> Hi Dragos,
>
> Could you try this pre-release version of Darshan and us know if it 
> works for you?
>
> ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-2.2.4-pre6.tar.gz
>
> The darshan-gen-* scripts will only work with mvapich2.
>
> I noticed an unrelated issue when trying to test this release on 
> Ranger, however.  I was not able to build a static executable using 
> mvapich2 (with or without darshan) because it could not find a static 
> version of the libverbs library. I was trying to generate a static 
> executable by just adding -static to the mpicc command line.  Maybe 
> there is an additional step needed to get a fully static executable?
>
> thanks,
> -Phil
>
> On 11/27/2012 10:25 AM, Phil Carns wrote:
>> Hi Dragos,
>>
>> Thanks for the bug report.  It looks like the 
>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc is just ordering the link 
>> arguments differently than darshan-gen-cc.pl expected.  We should be 
>> able to work around this without too much trouble. In terms of the 
>> perl code I think we just need to modify the regular expression to 
>> collect a "$link_cmd_prefix" in addition to a "$link_cmd_suffix" if 
>> anything appears in the link command line from the first '-' 
>> character up to the object name.  We can then just pass those 
>> arguments as is into the generated script. In this example the 
>> link_cmd_prefix would be:
>>
>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link 
>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>
>> I would like to see that particular mpicc script before making any 
>> changes, though, to make sure that we don't accidentally break 
>> something, but as (bad) luck would have it Ranger is in maintenance 
>> today.  We'll have a look at it tomorrow.
>>
>> thanks,
>> -Phil
>>
>> On 11/26/2012 03:27 PM, Dragos Constantin wrote:
>>> Hi Kevin,
>>> The problem is not with the argument parsing. This is what I get in 
>>> both cases:
>>>
>>> login4$ ./darshan-gen-cc.pl /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc 
>>> --output mpicc.darshan
>>> CC_from_link = gcc 
>>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link 
>>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>> CC_from_compile = gcc
>>> Error: cannot find matching CC from: gcc -c foo.c 
>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich 
>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>
>>> login4$ ./darshan-gen-cc.pl --output mpicc.darshan 
>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc
>>> CC_from_link = gcc 
>>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link 
>>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>> CC_from_compile = gcc
>>> Error: cannot find matching CC from: gcc -c foo.c 
>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich 
>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>
>>> As you can see:
>>>
>>> CC_from_compile = gcc
>>>
>>> but CC_from_link is not gcc and if I am not mistaken it should be 
>>> gcc. I just started to look at the script and you might know better 
>>> what is going on here.
>>>
>>> Thanks,
>>> Dragos
>>>
>>>
>>> Dragos Constantin, PhD
>>>
>>> Research Associate
>>> Department of Radiology
>>> Stanford University
>>> Lucas MRS Center
>>> 1201 Welch Rd., PS-055
>>> Stanford CA 94305
>>>
>>> Office: (650) 736-9961
>>> Fax: (650) 723-5795
>>>
>>> ----- Original Message -----
>>> From: "Kevin Harms" <harms at alcf.anl.gov>
>>> To: "Dragos Constantin" <dragos.constantin at stanford.edu>
>>> Cc: darshan-users at lists.mcs.anl.gov
>>> Sent: Monday, November 26, 2012 12:23:00 PM
>>> Subject: Re: [Darshan-users] Instrumenting statically-linked 
>>> applications
>>>
>>>
>>>    I think this might be a simple issue with argument parsing. Try 
>>> this instead:
>>>
>>>> ./darshan-gen-cc.pl --output mpicc.darshan 
>>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc
>>> kevin
>>>
>>> On Nov 26, 2012, at 2:16 PM, Dragos Constantin wrote:
>>>
>>>> Hi,
>>>> I've installed and configured darshan-2.2.3 on TACC Ranger in my 
>>>> user space. I have used gcc-4.4.5 (and mvapich-1.0.1).
>>>>
>>>> When I try to generate the MPI compiler scripts for 
>>>> statically-linked applications I get the following error:
>>>>
>>>> login4$ ./darshan-gen-cc.pl 
>>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc --output mpicc.darshan
>>>> Error: cannot find matching CC from: gcc -c foo.c 
>>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared 
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich 
>>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>>
>>>> I am not quite sure what triggered this. Any ideas how to quickly 
>>>> fix the issue? I will look at the perl script to see what is going 
>>>> on there.
>>>>
>>>> Thanks,
>>>> Dragos
>>>>
>>>>
>>>> Dragos Constantin, PhD
>>>>
>>>> Research Associate
>>>> Department of Radiology
>>>> Stanford University
>>>> Lucas MRS Center
>>>> 1201 Welch Rd., PS-055
>>>> Stanford CA 94305
>>>>
>>>> Office: (650) 736-9961
>>>> Fax: (650) 723-5795
>>>>
>>>> _______________________________________________
>>>> Darshan-users mailing list
>>>> Darshan-users at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>>> _______________________________________________
>>> Darshan-users mailing list
>>> Darshan-users at lists.mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>>
>> _______________________________________________
>> Darshan-users mailing list
>> Darshan-users at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>
>