[Darshan-users] Instrumenting statically-linked applications

Bill Barth bbarth at tacc.utexas.edu
Wed Nov 28 19:31:50 CST 2012 

Dragos,

Your directories are available on all the compute nodes on Ranger, so if
your darshan dynamic libs are in any of your directories, you should be
able to set your LD_LIBRARYPATH or the executable rpath appropriately to
point at your version of the darshan dynamic libraries.

Is there a reason you prefer the static version?

Bill.
--
Bill Barth, Ph.D., Director, HPC
bbarth at tacc.utexas.edu        |   Phone: (512) 232-7069
Office: ROC 1.435             |   Fax:   (512) 475-9445







On 11/28/12 6:20 PM, "Dragos Constantin" <dragos.constantin at stanford.edu>
wrote:

>Hi Phil,
>So, v2.2.4-pre6 works on both Ranger and Lonestar. I can confirm that
>darshan generates a log file. However, I am a bit confused because when
>I parse the log file it says my application did not open any file and in
>fact I am at least generating several output files. Maybe I have to
>configure something or supply some flags at compilation time so I can
>capture the full I/O load generated by my application.
>
>Do you think I should use the darshan-test application and see what the
>output looks like?
>
>You are right one cannot build 100% static executables on Ranger and
>Lonestar. However, the dynamic libs such as libverbs are installed on
>each compute node so, it is not an issue. What is more important is that
>the darshan lib and all my other libs are statically linked because they
>are not deployed system wide. In any case, I am thinking I should have
>seen in the darshan log files that some I/O activity occurred because of
>these dynamic libs.
>
>
>Thanks,
>Dragos
>
>
>
>Dragos Constantin, PhD
>
>Research Associate
>Department of Radiology
>Stanford University
>Lucas MRS Center
>1201 Welch Rd., PS-055
>Stanford CA 94305
>
>Office: (650) 736-9961
>Fax: (650) 723-5795
>
>On 11/28/2012 11:25 AM, Phil Carns wrote:
>> Hi Dragos,
>>
>> Could you try this pre-release version of Darshan and us know if it
>> works for you?
>>
>> ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-2.2.4-pre6.tar.gz
>>
>> The darshan-gen-* scripts will only work with mvapich2.
>>
>> I noticed an unrelated issue when trying to test this release on
>> Ranger, however.  I was not able to build a static executable using
>> mvapich2 (with or without darshan) because it could not find a static
>> version of the libverbs library. I was trying to generate a static
>> executable by just adding -static to the mpicc command line.  Maybe
>> there is an additional step needed to get a fully static executable?
>>
>> thanks,
>> -Phil
>>
>> On 11/27/2012 10:25 AM, Phil Carns wrote:
>>> Hi Dragos,
>>>
>>> Thanks for the bug report.  It looks like the
>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc is just ordering the link
>>> arguments differently than darshan-gen-cc.pl expected.  We should be
>>> able to work around this without too much trouble. In terms of the
>>> perl code I think we just need to modify the regular expression to
>>> collect a "$link_cmd_prefix" in addition to a "$link_cmd_suffix" if
>>> anything appears in the link command line from the first '-'
>>> character up to the object name.  We can then just pass those
>>> arguments as is into the generated script. In this example the
>>> link_cmd_prefix would be:
>>>
>>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link
>>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>>
>>> I would like to see that particular mpicc script before making any
>>> changes, though, to make sure that we don't accidentally break
>>> something, but as (bad) luck would have it Ranger is in maintenance
>>> today.  We'll have a look at it tomorrow.
>>>
>>> thanks,
>>> -Phil
>>>
>>> On 11/26/2012 03:27 PM, Dragos Constantin wrote:
>>>> Hi Kevin,
>>>> The problem is not with the argument parsing. This is what I get in
>>>> both cases:
>>>>
>>>> login4$ ./darshan-gen-cc.pl /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc
>>>> --output mpicc.darshan
>>>> CC_from_link = gcc
>>>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link
>>>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>>> CC_from_compile = gcc
>>>> Error: cannot find matching CC from: gcc -c foo.c
>>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich
>>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>>
>>>> login4$ ./darshan-gen-cc.pl --output mpicc.darshan
>>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc
>>>> CC_from_link = gcc
>>>> -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared -Wl,-rpath-link
>>>> -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib
>>>> CC_from_compile = gcc
>>>> Error: cannot find matching CC from: gcc -c foo.c
>>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich
>>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>>
>>>> As you can see:
>>>>
>>>> CC_from_compile = gcc
>>>>
>>>> but CC_from_link is not gcc and if I am not mistaken it should be
>>>> gcc. I just started to look at the script and you might know better
>>>> what is going on here.
>>>>
>>>> Thanks,
>>>> Dragos
>>>>
>>>>
>>>> Dragos Constantin, PhD
>>>>
>>>> Research Associate
>>>> Department of Radiology
>>>> Stanford University
>>>> Lucas MRS Center
>>>> 1201 Welch Rd., PS-055
>>>> Stanford CA 94305
>>>>
>>>> Office: (650) 736-9961
>>>> Fax: (650) 723-5795
>>>>
>>>> ----- Original Message -----
>>>> From: "Kevin Harms" <harms at alcf.anl.gov>
>>>> To: "Dragos Constantin" <dragos.constantin at stanford.edu>
>>>> Cc: darshan-users at lists.mcs.anl.gov
>>>> Sent: Monday, November 26, 2012 12:23:00 PM
>>>> Subject: Re: [Darshan-users] Instrumenting statically-linked
>>>> applications
>>>>
>>>>
>>>>    I think this might be a simple issue with argument parsing. Try
>>>> this instead:
>>>>
>>>>> ./darshan-gen-cc.pl --output mpicc.darshan
>>>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc
>>>> kevin
>>>>
>>>> On Nov 26, 2012, at 2:16 PM, Dragos Constantin wrote:
>>>>
>>>>> Hi,
>>>>> I've installed and configured darshan-2.2.3 on TACC Ranger in my
>>>>> user space. I have used gcc-4.4.5 (and mvapich-1.0.1).
>>>>>
>>>>> When I try to generate the MPI compiler scripts for
>>>>> statically-linked applications I get the following error:
>>>>>
>>>>> login4$ ./darshan-gen-cc.pl
>>>>> /opt/apps/gcc4_4/mvapich/1.0.1/bin/mpicc --output mpicc.darshan
>>>>> Error: cannot find matching CC from: gcc -c foo.c
>>>>> -I/opt/apps/gcc4_4/mvapich/1.0.1/include
>>>>> and: gcc -Wl,-rpath,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>>> -Wl,-rpath-link -Wl,/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib/shared
>>>>> -L/opt/apps/gcc4_4/mvapich/1.0.1/lib foo.o -o foo -lmpich
>>>>> -L/opt/ofed//lib64/ -libverbs -libumad -lpthread -lpthread -lrt
>>>>>
>>>>> I am not quite sure what triggered this. Any ideas how to quickly
>>>>> fix the issue? I will look at the perl script to see what is going
>>>>> on there.
>>>>>
>>>>> Thanks,
>>>>> Dragos
>>>>>
>>>>>
>>>>> Dragos Constantin, PhD
>>>>>
>>>>> Research Associate
>>>>> Department of Radiology
>>>>> Stanford University
>>>>> Lucas MRS Center
>>>>> 1201 Welch Rd., PS-055
>>>>> Stanford CA 94305
>>>>>
>>>>> Office: (650) 736-9961
>>>>> Fax: (650) 723-5795
>>>>>
>>>>> _______________________________________________
>>>>> Darshan-users mailing list
>>>>> Darshan-users at lists.mcs.anl.gov
>>>>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>>>> _______________________________________________
>>>> Darshan-users mailing list
>>>> Darshan-users at lists.mcs.anl.gov
>>>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>>>
>>> _______________________________________________
>>> Darshan-users mailing list
>>> Darshan-users at lists.mcs.anl.gov
>>> https://lists.mcs.anl.gov/mailman/listinfo/darshan-users
>>
>>
>
>_______________________________________________
>Darshan-users mailing list
>Darshan-users at lists.mcs.anl.gov
>https://lists.mcs.anl.gov/mailman/listinfo/darshan-users

More information about the Darshan-users mailing list