[MPICH] MPLMD (Multiple Programming Languages Multiple Data)

[Ralph M. Butler]

Perhaps I should not comment since I cannot address your main problem
with Java, but I will toss in 2 cents anyway.  I occassionally develop
code in which a single app uses a combination of Python and C.  Just
last week I wrote one on linux which has multiple P-threads (as servers)
in a C program that uses MPI_Send/Recv on 3 different communicators.
The Python portions of the app have access to the MPI functions as well.
I am still coding/debugging that app, but believe that everything is
going to be OK.  My major concern going into this particular app was
that I would encounter bugs in the MPICH multi-threading support since
that is still relatively new.  But, so far things have gone fine.

A couple of years ago I examined some of the open Python wrappers for
MPI, but found that they were never quite what I wanted.  So, I just
roll my own whenever I need something.  I generally send Python
objects as pickled objects that can travel as byte strings.
So, in Python, I can do something like this:
    pickledMsg = cPickle.dumps(obj)
    mympi.Send(pickledMsg,toRank,tag)

Here is a sample wrapper to support that:

static PyObject *Send(PyObject *self, PyObject *pArgs)
{
    int msglen,torank,tag;
    char *msg;

    if ( ! PyArg_ParseTuple(pArgs,"s#ii",&msg,&msglen,&torank,&tag))
    {
	printf("Send: invalid args\n");
	return NULL;
    }
    MPI_Send(msg,msglen,MPI_BYTE,torank,tag,MPI_COMM_WORLD);
    return Py_BuildValue("");  /* None */
}

--ralph

> Date: Sat, 17 Dec 2005 10:28:57 +0100
> From: Koehne Kai <Kai.Koehne at student.hpi.uni-potsdam.de>
> To: mpich-discuss at mcs.anl.gov
> Subject: [MPICH] MPLMD (Multiple Programming Languages Multiple Data)
>
> Hello,
>
> first of all sorry if the question sounds stupid to you, or if this is not the right place to ask. I am pretty new to MPI in general, and could not find a more specific discussion board ...
>
> The Multiple Programs Multiple Data (MPMD) paradigm is pretty well documented for MPICH. However, I am trying to let multiple programs, written in different programming languages (Java and Python), to communicate. I tried to achieve that with two MPI wrapper libraries, namely mpiJava and pypar.
>
> However, I failed: When combining small Python and Java demo programs in one MPICH procgroup file, the Java programs seem to hang up in the MPI initialization call.
>
> It might be that this problem is specific to the mpiJava wrapper implementation. However, I am a bit astonished that the whole idea of combining MPI programs from different programming languages is hardly ever explained in the literature. Have you had experience with such a setting? And can you recommend me a specific Java wrapper implementation that is known to communicate nicely with other wrapper implementations?
>
> Regards,
>
> Kai Koehne
> Hasso-Plattner-Instute
> University of Potsdam
> Germany
>
>
>