[Darshan-users] how many compilers and MPI libraries can a single build of Darshan support?

[Phil Carns]

On 03/07/2013 02:38 PM, David Shrader wrote:
> Hello All,
>
> I've been playing around a lot with Darshan the last couple of days 
> and I am currently trying to find out how much a single build of 
> Darshan can support in terms of other compilers and MPI libraries. The 
> documentation has this statement if Darshan's CC is based on a GNU 
> compiler, " Once Darshan has been installed, it should be capable of 
> instrumenting executables built with GNU, Intel, and PGI compilers." I 
> have found that not only will Darshan work with GNU, Intel, and PGI 
> compilers, it seems to work with any version of these compilers (well, 
> any versions that I have in production right now) as well as Pathscale 
> compilers. Is this to be expected?
>
> For example, I have built Darshan with Open MPI 1.6.4 that was built 
> using the GNU system compilers (v4.4.6). This single build of Darshan 
> is able to operate on a binary that was built with Open MPI 1.6.4 
> based on the following compilers: gcc 4.6.1, gcc 4.7.0, gcc 4.7.2, 
> intel 11.1.072, intel 12.1.2, intel 12.1.5, pgi 9.0-3, pgi 10.9, pgi 
> 12.6, pgi 12.10 and pathscale 4.0. Meaning, I have builds of Open MPI 
> 1.6.4 compatible with each of these compilers and I use them to build 
> my application. No matter what version of Open MPI 1.6.4 I use to 
> compile my application, I can get Darshan output from running it 
> through libdarshan.so (that was built using Open MPI 1.6.4 based on 
> the system GNU compilers). I have done a similar test with mvapich2 
> 1.8. This is absolutely amazing to me and I would like to know if it 
> is expected.
>

Hi David,

This much is indeed expected, though it is always great to hear some 
confirmation!  If Darshan itself is built with a GNU compiler, then the 
resulting library will work with executables built with all of the 
compilers you listed above (GNU, Intel, PGI, Pathscale) as well as Cray 
and IBM compilers.  I've never tested it with LLVM, but in theory it 
should work fine with that one too.  We are fortunate that the linking 
is compatible across all of these compilers :-)

> A related question to that is what can a single Darshan build support 
> in terms of MPI libraries? I have found that my Open MPI 1.6.4 build 
> will not support mvapich2 compiled binaries and vice versa. But, I 
> have found that my mvapich2 1.8 build supports mvapich2 1.7, 1.8 and 
> 1.9a2 compiled binaries. Is this expected as well? Since only Open MPI 
> 1.6.4 runs well with Darshan out of the box, I don't have a similar 
> test there.

This part is harder to answer.  The problem is that various MPI 
implementations will not necessarily use the same underlying types or 
predefined values for all of the MPI data structures.  They aren't 
required to be binary compatible.

I'm not an expert on this and I can't guarantee anything, but I would 
guess that the major MPI implementations probably have pretty stable 
ABIs across point releases.  All bets are off across implementations, 
though.

 From a packaging/deployment point of view, what this means is that you 
don't have to switch Darshan builds when users select different 
underlying compilers, but you probably do need to switch
when they select different MPI implementations.

thanks,
-Phil

> In short, I am amazed at the field that a single build of Darshan 
> seems to be able to support. I'm curious to know if there is any 
> guidance on what the expected edges of that field are.
>
> Again, thank you for all of the help!
> David
>