Fw: [mpich-discuss] exit code -1073741819
Tadeusz J Liszka
tad at tx.altair.com
Tue Mar 25 15:13:06 CDT 2008
Right - on Windows this would mean more than 10 different binary libs,
which is not practical. But it seems to me that we should be able to
build the library from the source, using any reasonable compiler
settings. We'll try that.
Jayesh Krishna wrote:
> Hi,
>
>>> I do not agree with your suggestion: "since you can set the different
>>>
> options in the compiler to achieve your need"...
> As developers of MPICH2 we would like our lib to support all
> available fortran compilers with least/no effort from the user (MPI dev).
> However, providing a binding library for all the possible combinations of
> compiler options would not be the best idea. If you are using the Intel V
> fortran compiler try using fmpich2.lib.
> Are you facing difficulties in compiling your MPI program too ?
>
> Regards,
> Jayesh
> -----Original Message-----
> From: Tadeusz J Liszka [mailto:tad at tx.altair.com]
> Sent: Tuesday, March 25, 2008 12:31 PM
> To: Linfa Zhu; Jayesh Krishna; trymelz trymelz
> Cc: mpich-discuss at mcs.anl.gov
> Subject: Re: Fw: [mpich-discuss] exit code -1073741819
>
> Hi,
>
> I do not agree with your suggestion: "since you can set the different
> options in the compiler to achieve your need". It is not always possible to
> change compiler options (because of various libraries already used in the
> project), so we need to be able to compile mpich in the proper way.
> But I can see that most, if not all, what is needed for this, is already
> present in the source.
>
> I could not find proper bindings for MPI_SEND - maybe I was not looking in
> the proper places:
>
> As MPI_SEND uses void* for the first argument, it requires (for some
> compilers, at least on Windows) two alternate bindings. One of them should
> have 7 arguments - to handle all calls where the void argument is integer or
> real array - and the second one with 8 arguments, to handle hidden argument
> (==string length). When this argument is at the end, the problem is (almost)
> trivial, but it is quite difficult to handle when the compiler insists on
> appending this hidden argument immediately after the string pointer. In
> STDCALL Windows binding, the name mangling
> (mpi_send at 28/mpi_send at 32) resolves the issue, in CDECL binding the problem
> is much more tricky to resolve. Fortunately we need STDCALL mode.
>
> Thus we need
> extern FORT_DLL_SPEC void FORT_CALL MPI_SEND( void*, MPI_Fint *, MPI_Fint *,
> MPI_Fint *, MPI_Fint *, MPI_Fint *, MPI_Fint * ); as well as extern
> FORT_DLL_SPEC void FORT_CALL MPI_SEND( char* FORT_MIXED_LEN(str_len),
> MPI_Fint *, MPI_Fint *, MPI_Fint *, MPI_Fint *, MPI_Fint *, MPI_Fint *
> FORT_END_LEN_DECL );
>
> I do not say that this is not present in the source - just that in the time
> I spent on it I could not locate it, and I found plenty other places where
> these dummy arguments are present. Moreover I am sure that I missed
> something, because apparently the MPI_SEND at 32 entry is present in the
> library, and I did not locate exact place where it comes from.
>
> Assuming that the second binding is present somewhere in the source, it is
> my understanding that all we need to do is clean recompile code with proper
> option set for USE_FORT_MIXED_STR_LEN, and no code changes in mpich source
> are required.
>
> regards
>
> Tadeusz
>
> Linfa Zhu wrote:
>
>>
>> ----- Original Message -----
>> *From:* trymelz trymelz <mailto:trymelz at yahoo.com>
>> *To:* linfa at tx.altair.com <mailto:linfa at tx.altair.com>
>> *Sent:* Tuesday, March 25, 2008 11:20 AM
>> *Subject:* Fw: [mpich-discuss] exit code -1073741819
>>
>>
>>
>> ----- Forwarded Message ----
>> From: Jayesh Krishna <jayesh at mcs.anl.gov <mailto:jayesh at mcs.anl.gov>>
>> To: trymelz trymelz <trymelz at yahoo.com <mailto:trymelz at yahoo.com>>
>> Cc: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>> Sent: Tuesday, March 25, 2008 10:43:15 AM
>> Subject: RE: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>> As I mentioned in one of my previous emails, "The 8th argument is to
>> handle the fortran CHARACTER arrays" (The 8th argument is not exposed
>> to the user. It is used internally by the lib.) If you would like to
>> play around with the fortran binding interface (Which we suggest that
>> you do not since you can set the different options in the compiler to
>> achieve your need. And this change might prevent you from upgrading to
>> later releases.) I suggest that you take a look at
>> "src\binding\f77\mpi_fortimpl.h"
>>
>> Regards,
>>
>> Jayesh
>>
>>
>> ----------------------------------------------------------------------
>> --
>> *From:* owner-mpich-discuss at mcs.anl.gov
>> [mailto:owner-mpich-discuss at mcs.anl.gov] *On Behalf Of *trymelz
>> trymelz
>> *Sent:* Tuesday, March 25, 2008 10:01 AM
>> *To:* mpich-discuss at mcs.anl.gov
>> *Subject:* Re: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>>
>>
>>
>> I checked the f77 binding of mpi_send. Here is what I got
>>
>>
>>
>> $ grep -i MPI_SEND sendf.c | grep -v define | grep -v pragma extern
>> FORT_DLL_SPEC void FORT_CALL MPI_SEND( void*, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC
>> void FORT_CALL mpi_send__( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send_( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL pmpi_send_( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL MPI_SEND( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send__( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send_( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL MPI_SEND( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send__( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); extern FORT_DLL_SPEC void
>> FORT_CALL mpi_send_( void*, MPI_Fint *, MPI_Fint *, MPI_Fint *,
>> MPI_Fint *, MPI_Fint *, MPI_Fint * ); #undef MPI_Send FORT_DLL_SPEC
>> void FORT_CALL mpi_send_ ( void*v1, MPI_Fint *v2, MPI_Fint *v3,
>> MPI_Fint *v4, MPI_Fint *v5, MPI_Fint *v6, MPI_Fint *ierr ){
>> *ierr = MPI_Send( v1, *v2, (MPI_Datatype)(*v3), *v4, *v5,
>> (MPI_Comm)(*v6) );
>>
>> I could not find the 8th argument from above. Did I look into a wrong
>> source file (src/binding/f77/sendf.c)?
>>
>>
>>
>> I understand that changing the setup of string length argument for my
>> program could resolve this issue. The bad news is that I was told it
>> can not be changed due to other issues. So I am wondering how I can
>> modify the MPICH2 code (move the 8th arguments to the 2nd place) to
>> fit my needs. Thanks.
>>
>>
>> Linfa
>>
>> ----- Original Message ----
>> From: Jayesh Krishna <jayesh at mcs.anl.gov>
>> To: trymelz trymelz <trymelz at yahoo.com>
>> Cc: mpich-discuss at mcs.anl.gov
>> Sent: Friday, March 21, 2008 9:24:30 AM
>> Subject: RE: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>> MPI_SEND() in fortran takes 7 arguments. The 8th argument is to
>> handle the fortran CHARACTER arrays.
>>
>> Regards,
>> Jayesh
>>
>> ----------------------------------------------------------------------
>> --
>> *From:* owner-mpich-discuss at mcs.anl.gov
>> [mailto:owner-mpich-discuss at mcs.anl.gov] *On Behalf Of *trymelz
>> trymelz
>> *Sent:* Thursday, March 20, 2008 6:16 PM
>> *To:* mpich-discuss at mcs.anl.gov
>> *Subject:* Re: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>>
>> I did not see MPI_SEND taking 8 arguments, how come it generates
>> MPI_SEND at 32 ?
>>
>> Linfa
>>
>> ----- Original Message ----
>> From: Jayesh Krishna <jayesh at mcs.anl.gov>
>> To: trymelz trymelz <trymelz at yahoo.com>
>> Cc: mpich-discuss at mcs.anl.gov
>> Sent: Thursday, March 20, 2008 3:44:25 PM
>> Subject: RE: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>> The fortran bindings for MPI funcs is in src/binding/f77 (Note that
>> these source files are generated by scripts).
>>
>> Regards,
>> Jayesh
>>
>> ----------------------------------------------------------------------
>> --
>> *From:* owner-mpich-discuss at mcs.anl.gov
>> [mailto:owner-mpich-discuss at mcs.anl.gov] *On Behalf Of *trymelz
>> trymelz
>> *Sent:* Thursday, March 20, 2008 3:38 PM
>> *To:* mpich-discuss at mcs.anl.gov
>> *Subject:* Re: [mpich-discuss] exit code -1073741819
>>
>> Hi,
>>
>>
>>
>> By specifying character length as 11, the program behaviors the same
>> as 15 since the order of arguments is messed up.
>>
>>
>>
>> Do you know where is the protype or definition in the source code of
>> MPI_SEND/RECV? Thanks.
>>
>>
>>
>>
>>
>> Linfa
>>
>>
>>
>> ----- Original Message ----
>> From: Anthony Chan <chan at mcs.anl.gov>
>> To: mpich-discuss at mcs.anl.gov
>> Cc: Jayesh Krishna <jayesh at mcs.anl.gov>
>> Sent: Thursday, March 20, 2008 2:33:51 PM
>> Subject: Re: [mpich-discuss] exit code -1073741819
>>
>>
>> What happens if you specify character length as 11 instead of 15 in
>> your MPI_Sned and MPI_Recv ?
>>
>> On Thu, 20 Mar 2008, trymelz trymelz wrote:
>>
>>
>>> Hi,
>>>
>>> Sure. After changing the calling convention to default in Intel
>>>
>> Compiler, I successfully linked with fmpich2.lib. But again, it runs
>> fine when the string length is passed at the end of argument list, and
>> it crashes when the string length is passed after the string pointer.
>>
>>> Probably, more MPICH2 FORTRAN libraries are needed to be provided.
>>>
>>> Linfa
>>>
>>> ----- Original Message ----
>>> From: Jayesh Krishna <jayesh at mcs.anl.gov
>>> <mailto:jayesh at mcs.anl.gov>>
>>> To: trymelz trymelz <trymelz at yahoo.com <mailto:trymelz at yahoo.com>>
>>> Cc: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>>> Sent: Thursday, March 20, 2008 2:08:46 PM
>>> Subject: RE: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi,
>>> Can you also try compiling your program with fmpich2.lib ?
>>>
>>> Regards,
>>> Jayesh
>>>
>>>
>>>
>>> From: trymelz trymelz [mailto:trymelz at yahoo.com
>>>
>> <mailto:trymelz at yahoo.com>]
>>
>>> Sent: Thursday, March 20, 2008 2:06 PM
>>> To: Jayesh Krishna; mpich-discuss at mcs.anl.gov
>>>
>> <mailto:mpich-discuss at mcs.anl.gov>
>>
>>> Subject: Re: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi
>>>
>>> I have no problem to compile & run the sample fpi.f using
>>>
>> fmpich2s.lib. Actually, I have identified the culprit: MPI_SEND/RECV
>> work find with integer, but have problem with string.
>>
>>> The main reason I beleive is that, in Fortran, different compiler
>>>
>> treats the string arguments differently: either add the string length
>> as a invisible argument immediately after the string pointer, or the
>> string lengh is added as a invisible arguments at the end of arguments
>> list. So in my understanding MPI_SEND/RECV will have 28 byte of
>> arguments for integer, but 32 byte of arguments for string because the
>> invisible string length argument.
>>
>>> It will cause crash if the positions of the invisible string length
>>>
>> arguments are different in the .obj file and in fmpich2s.lib. The
>> following program dimenstrate this
>>
>>> program main
>>> implicit none
>>> #include <mpif.h>
>>> integer status(MPI_STATUS_SIZE)
>>> character * 15 pcomand
>>> integer MYID,mpimodbg,ierr
>>> ierr=MPI_SUCCESS
>>> call MPI_INIT(ierr)
>>> call MPI_COMM_RANK( MPI_COMM_WORLD, MYID, ierr ) if(MYID .eq. 1)then
>>> mpimodbg=99
>>> pcomand='hello world'
>>> call MPI_SEND(mpimodbg,1,MPI_INTEGER,0
>>> * ,11,MPI_COMM_WORLD,ierr)
>>> write(*,*)'node',MYID,pcomand,mpimodbg
>>> call MPI_SEND(pcomand,15,MPI_CHARACTER,0
>>> * ,11,MPI_COMM_WORLD,ierr)
>>> write(*,*)'node',MYID,pcomand,mpimodbg
>>> elseif(MYID.eq.0)then
>>> mpimodbg=0
>>> pcomand='hello'
>>> call MPI_RECV(mpimodbg,1,MPI_INTEGER,1
>>> * ,11,MPI_COMM_WORLD,status,ierr)
>>> write(*,*)'node',MYID,pcomand,mpimodbg
>>> call MPI_RECV(pcomand,15,MPI_CHARACTER,1
>>> * ,11,MPI_COMM_WORLD,status,ierr)
>>> write(*,*)'node',MYID,pcomand,mpimodbg
>>> endif
>>> call MPI_FINALIZE(ierr)
>>> end
>>>
>>> When compiled with the string length as a invisible argument
>>>
>> immediately after the string pointer (by default in Intel Compiler):
>>
>>> $ "/cygdrive/c/Program Files/MPICH2/bin/mpiexec.exe" -n 3
>>>
>> ./Debug/fortestargs.exe
>>
>>> node 0 hello 99
>>> node 1 hello world 99
>>> forrtl: severe (157): Program Exception - access violation
>>> Image PC Routine Line Source
>>> fmpich2s.dll 1000507F Unknown Unknown Unknown
>>> fortestargs.exe 004A783D Unknown Unknown Unknown
>>> fortestargs.exe 0044AA53 Unknown Unknown Unknown
>>> fortestargs.exe 0044A81D Unknown Unknown Unknown
>>> kernel32.dll 7C816FD7 Unknown Unknown Unknown
>>> forrtl: severe (157): Program Exception - access violation
>>> Image PC Routine Line Source
>>> fmpich2s.dll 1000492B Unknown Unknown Unknown
>>> fortestargs.exe 004A783D Unknown Unknown Unknown
>>> fortestargs.exe 0044AA53 Unknown Unknown Unknown
>>> fortestargs.exe 0044A81D Unknown Unknown Unknown
>>> kernel32.dll 7C816FD7 Unknown Unknown Unknown
>>> job aborted:
>>> rank: node: exit code[: error message]
>>> 0: abyss: 157: process 0 exited without calling finalize
>>> 1: abyss: 157: process 1 exited without calling finalize
>>> 2: abyss: 123
>>>
>>> When compiled with the string length as a invisible argument at the
>>>
>> end of arguments list:
>>
>>> $ "/cygdrive/c/Program Files/MPICH2/bin/mpiexec.exe" -n 3
>>>
>> ./Debug/fortestargs.exe
>>
>>> node 1 hello world 99
>>> node 1 hello world 99
>>> node 0 hello 99
>>> node 0 hello world 99
>>>
>>>
>>> Thanks all for your help.
>>>
>>> Linfa
>>>
>>>
>>> ----- Original Message ----
>>> From: Jayesh Krishna <jayesh at mcs.anl.gov
>>> <mailto:jayesh at mcs.anl.gov>>
>>> To: trymelz trymelz <trymelz at yahoo.com <mailto:trymelz at yahoo.com>>
>>> Cc: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>>> Sent: Thursday, March 20, 2008 9:15:21 AM
>>> Subject: RE: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi,
>>> Can you compile & run the sample fpi.f (provided in
>>>
>> MPICH2\examples in your installation dir) using fmpich2s.lib ?
>>
>>> Regards,
>>> Jayesh
>>>
>>>
>>>
>>>
>>> From: owner-mpich-discuss at mcs.anl.gov
>>>
>> <mailto:owner-mpich-discuss at mcs.anl.gov>
>> [mailto:owner-mpich-discuss at mcs.anl.gov
>> <mailto:owner-mpich-discuss at mcs.anl.gov>] On Behalf Of trymelz trymelz
>>
>>> Sent: Wednesday, March 19, 2008 6:30 PM
>>> To: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>>> Subject: Re: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi
>>>
>>> The MPI program is too big to be sent. Here is what I got from the
>>>
>> debugger
>>
>>> Before calling PMPI_ISEND:
>>> Symbol Value Type
>>> buf(1) 3 INTEGER(4)
>>> cnt 1 INTEGER(4)
>>> datatype 1275069467 INTEGER(4)
>>> dest 0 INTEGER(4)
>>> tag 24 INTEGER(4)
>>> comm 1140850688 INTEGER(4)
>>> reqs(1) 1243468 INTEGER(4)
>>> ierr 0 INTEGER(4)
>>>
>>> call PMPI_ISEND(buf,cnt,datatype,dest,tag,comm,reqs(1),ierr)
>>>
>>> Unhandled exception at 0x003b41c9 in EXEC.exe:0xC0000005:
>>> Access violation writing location 0x00000000
>>>
>>> After the break at the crash, all the arguments becomes "undefined
>>>
>> address".
>>
>>> When I check the library, I got
>>> $ nm fmpich2s.lib | grep MPI_SEND@
>>> 00000000 T _MPI_SEND at 28
>>> 00000000 I __imp__MPI_SEND at 28
>>> 00000000 T _MPI_SEND at 32
>>> 00000000 I __imp__MPI_SEND at 32
>>> 00000000 T _PMPI_SEND at 28
>>> 00000000 I __imp__PMPI_SEND at 28
>>> 00000000 T _PMPI_SEND at 32
>>> 00000000 I __imp__PMPI_SEND at 32
>>>
>>> I am wondering why there are two definitions: _MPI_SEND at 28 and
>>>
>> _MPI_SEND at 32. And I could not find corresponding source code for
>> _MPI_SEND at 32. Any comments? Thanks.
>>
>>> Linfa
>>>
>>>
>>>
>>> ----- Original Message ----
>>> From: Jayesh Krishna <jayesh at mcs.anl.gov
>>> <mailto:jayesh at mcs.anl.gov>>
>>> To: trymelz trymelz <trymelz at yahoo.com <mailto:trymelz at yahoo.com>>
>>> Cc: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>>> Sent: Wednesday, March 19, 2008 2:32:59 PM
>>> Subject: RE: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi,
>>> Can you send us your MPI program ?
>>> You can also try to debug your program by setting the error handler
>>>
>> to MPI_ERRORS_RETURN (MPI_Comm_set_errhandler() ) & using
>> MPI_Error_string() to get the description of the error code.
>>
>>> Regards,
>>> Jayesh
>>>
>>>
>>>
>>>
>>> From: owner-mpich-discuss at mcs.anl.gov
>>>
>> <mailto:owner-mpich-discuss at mcs.anl.gov>
>> [mailto:owner-mpich-discuss at mcs.anl.gov
>> <mailto:owner-mpich-discuss at mcs.anl.gov>] On Behalf Of trymelz trymelz
>>
>>> Sent: Wednesday, March 19, 2008 2:11 PM
>>> To: mpich-discuss at mcs.anl.gov <mailto:mpich-discuss at mcs.anl.gov>
>>> Subject: [mpich-discuss] exit code -1073741819
>>>
>>>
>>> Hi all,
>>>
>>> I got an strange error when I run my parallel program.
>>>
>>> job aborted:
>>> rank: node: exit code[: error message]
>>> 0: abyss: -1073741819: process 0 exited without calling finalize
>>> 1: abyss: -1073741819: process 1 exited without calling finalize
>>> 2: abyss: -1073741819: process 2 exited without calling finalize
>>>
>>> My program works as expected under Linux. But when I porting it to
>>>
>> windows I got above error. It is compiled by VC8 and Intel FORTRAN 10.0.
>>
>>> I didn't built my own MPICH2, but link my program with fmpich2s.lib
>>>
>> and mpi.lib which were installed from mpich2-1.0.6p1-win32-ia32.msi.
>>
>>> This error happened inside the call to MPI_SEND. Does anyone have an
>>>
>> idea about this error?
>>
>>> Linfa
>>>
>>>
>>>
>>> Be a better friend, newshound, and know-it-all with Yahoo! Mobile.
>>>
>> Try it now.
>>
>>>
>>>
>>>
>>> Looking for last minute shopping deals? Find them fast with Yahoo!
>>>
>> Search.
>>
>>>
>>>
>>>
>>> Looking for last minute shopping deals? Find them fast with Yahoo!
>>>
>> Search.
>>
>>>
>>>
>> ______________________________________________________________________
>> ______________
>>
>>> Looking for last minute shopping deals?
>>> Find them fast with Yahoo! Search.
>>>
>> http://tools.search.yahoo.com/newsearch/category.php?category=shopping
>>
>>
>>
>> ----------------------------------------------------------------------
>> -- Be a better friend, newshound, and know-it-all with Yahoo! Mobile.
>> Try it now.
>> <http://us.rd.yahoo.com/evt=51733/*http://mobile.yahoo.com/;_ylt=Ahu06
>> i62sR8HDtDypao8Wcj9tAcJ%20>
>>
>>
>> ----------------------------------------------------------------------
>> -- Never miss a thing. Make Yahoo your homepage.
>> <http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs>
>>
>>
>> ----------------------------------------------------------------------
>> -- Never miss a thing. Make Yahoo your homepage.
>> <http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs>
>>
>>
>> ----------------------------------------------------------------------
>> -- Never miss a thing. Make Yahoo your homepage.
>> <http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs>
>>
>
>
--
## # ##### ## ### ### Tadeusz J. Liszka, Ph.D.
# # # # # # # # # Altair Engineering Inc.
#### # # #### # ### http://www.tx.altair.com
# # #### # # # ### # # tel. (512)467-0618 #526
More information about the mpich-discuss
mailing list