[mpich2-dev] Missing symbols when configuring with --disable-f77 --disable-f90

[Dave Goodell]

On Aug 3, 2009, at 3:31 PM, Lisandro Dalcin wrote:

> On Mon, Aug 3, 2009 at 3:44 PM, Dave Goodell<goodell at mcs.anl.gov>  
> wrote:
>> On Aug 3, 2009, at 1:23 PM, Lisandro Dalcin wrote:
>>
>> AFAIK isn't a good way to provide these functions when f77/f90 are  
>> disabled.
>>  Furthermore I don't think that we can write safe "dummy"  
>> implementations
>> since we aren't actually checking the Fortran build environment.
>>
>
> What about providing the symbols, but make the call generate an error?
> Other MPI implementations do this, for example OpenMPI without ROMIO;
> also Microsoft MPI precisely for MPI_Type_create_f90_XXX ...

We could probably do something like that eventually.  Let me spend  
more time looking at it.  I suspect it won't be that straightforward  
because of the code generation involved in the src/binding/f90  
directory.

>> IMO the right fixes in this case are on either the user's side (don't
>> disable f77/f90)
>
> Many people out there are using a pre-built MPICH2. They do not have
> control about how MPICH2 is built, so they cannot make the choice of
> enabling/disabling Fortran. They just have to live with what the
> sysadmin or the Linux distro provides...

I understand that.  I was just pointing it out as one possible  
solution in some cases.  Also, if we can't rely on the user to change  
their MPI implementation then we can't rely on any fix that is  
provided on the MPICH2 side for a year or so, which is why I  
recommended an mpi4py-side solution.

> So we cannot rely on users, next it seems that mpi4py's author is in
> charge of alleviating users for these issues :-) . Despite you put the
> work on my shoulders, I'm fine with that but ...
>
>> or on mpi4py's side (add configure checks and possibly a
>> dummy library of your own).
>>  You could alter the bindings for those 5 or so
>> missing functions to throw an exception if the user calls those  
>> functions in
>> python and they are not available in the underlying MPI library.
>>
>
> mpi4py already provide the mechanism to do so. The problem is that
> checking for every possible missing bit of any MPI implementation is
> really time consumming. So I've decided to not enable it by default.
> Perhaps I should change my mind and check for every of the 555 MPI
> symbols (that's my count for what I'm using in mpi4py)...

I wasn't suggesting that you check for all 555 functions.  Rather, I  
was trying to suggest that you check for these 5 functions, which  
should take quite a bit less time.

> Dave, I could live with a MPICH2 that does not provide the missing
> calls. In fact, I live just fine with MPICH2 do not providing the I/O
> calls if ROMIO not was configured... I can live with that because I
> can make (BTW, non-standard) compile time choices "#if defined(MPICH2)
> && !defined(ROMIO_VERSION)".

You could also run a configure-style link check for a few key MPI-I/O  
symbols and assume that MPI-I/O isn't available if any of them are  
missing.  Generally speaking, specific feature checks are a much safer  
way to determine this sort of thing.

> What I cannot buy easily is MPICH2 providing a "mpi.h" header saying
> that "MPI_<Something>" is there, but the library does not have
> "MPI_<Something>"... If MPICH2 would provide at least a way to use the
> C preprocessor to handle at compile time the missing stuff...

We could probably do this, although something about it rubs me the  
wrong way.

> Please, Dave, try to take my place for a bit of time... Suppose you
> are trying to build a wrapper for a language other than
> C/C++/Fortran... It is really really hard to do a good job. You have
> to write workarounds for whatever interpretation of corner cases in
> the standard the MPI implementation takes. You have to support plus
> MPI-1 only, MPI-1 and bits of MPI-2, and full MPI-2 calls. Next you
> have to cope with dynamic linking issues (and the do-nothing policy of
> Open MPI folks). Now add to the equation that every MPI implementation
> out there will decide what to provide and what not to provide in the
> MPI header/library, or use funny calling conventions (MS MPI on Win32,
> with BTW does not even follow 100% their own conventions)... and then
> you get really lost and angry because you want to do it well, but it
> is SO hard...

Sorry to be gruff about this earlier, that wasn't my intention.  I was  
trying to answer your question in a hurry and give you my views on  
this issue.

I can understand where you are coming from because we encounter many  
of the same issues at the next level down with operating systems and  
networking stacks.  This platform supports weak symbols but that one  
doesn't, this system has pthreads and that has windows threads, this  
system implements snprintf safely and this one clobbers your null  
byte, etc...  Unfortunately the best we can do is to check for these  
issues and work around them as we learn about them.

> In short, Dave, is there any chance that if MPICH2 cannot provide a
> implementation for these calls, then do one of:
>
> Option 1) Make the symbol available, but make it fail (I mean, MPI
> error) at runtime.

I will investigate what it would take to do this and get back to you.

> Option 2) Provide a macro in "mpi.h" (or an additional header that can
> be conditionally #include'd?) advertising the missing call, and
> perhaps even to not list the call in the "mpi.h" header?


We might be able to do this too (removing the prototypes, not adding  
feature test macros).  Let me look at the stub function solution and  
compare the amount of work involved.

There is also another possibility here.  If you don't actually need  
interoperability with fortran you could just always skip these symbols  
in your wrapper.

Also, remember that neither of these options deal with the users who  
are stuck on 1.0.8, 1.1.0, or 1.1.1 for the next year or so for  
various reasons.  You will still probably need to fix this on your end  
regardless of what we do.

-Dave