<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<STYLE type=text/css>DIV {
MARGIN: 0px
}
</STYLE>
<META content="MSHTML 6.00.6000.16525" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=004544915-14112007><FONT face=Arial
color=#0000ff size=2>Hi,</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=004544915-14112007><FONT face=Arial
color=#0000ff size=2> KIlling an MPI process results in the process manager
aborting all the MPI processes associated with the current job. This error is
not an MPI error (The errhandler associated with a communicator can only handle
MPI errors). The error message is then printed by the process manager (SMPD in
the case of windows). </FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=004544915-14112007><FONT face=Arial
color=#0000ff size=2> I believe what you need is an MPI library
implementation where you could kill one of the MPI processes and still have the
MPI job running (the remaining MPI processes running). You could modify the
source code of SMPD to do that.</FONT> </SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=004544915-14112007><FONT face=Arial
color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=004544915-14112007><FONT face=Arial
color=#0000ff size=2>(PS: Strictly speaking this level of fault tolerance should
be handled, if possible, at the process manager/library level -- not in the user
code.)</FONT></SPAN></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><SPAN class=004544915-14112007></SPAN><FONT face=Arial><FONT
color=#0000ff><FONT size=2>R<SPAN
class=004544915-14112007>egards,</SPAN></FONT></FONT></FONT></DIV>
<DIV><FONT><FONT color=#0000ff><FONT size=2><SPAN
class=004544915-14112007></SPAN></FONT></FONT></FONT><SPAN
class=004544915-14112007></SPAN><FONT face=Arial><FONT color=#0000ff><FONT
size=2>J<SPAN
class=004544915-14112007>ayesh</SPAN></FONT></FONT></FONT><BR></DIV>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> AGPX [mailto:agpxnet@yahoo.it]
<BR><B>Sent:</B> Wednesday, November 14, 2007 7:39 AM<BR><B>To:</B> Jayesh
Krishna<BR><B>Cc:</B> mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> Re: [MPICH]
Error handling issue<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: courier,monaco,monospace,sans-serif">
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="FONT-FAMILY: times new roman,new york,times,serif">Hi,<BR><BR>I abort the
process by killing the process (from Task Manager). Basically, my application
(on the so called 'main machine', ID = 0) distribute his calculation on various
machines (called the 'evaluators'). When an evaluator abort for any reason
(could be also a blackout) I need to handle this situation in order to delegate
his calculation to another evaluator, so that I can avoid to lost calculations
already done by the other evaluators. Actually, when an evaluator abort (I have
tried to kill the process), the main process (ID = 0) abort with the error
message described and this is a serious problem for me.<BR></SPAN><BR
style="FONT-FAMILY: times new roman,new york,times,serif"><SPAN
style="FONT-FAMILY: times new roman,new york,times,serif">Best
regards,</SPAN><BR style="FONT-FAMILY: times new roman,new york,times,serif"><BR
style="FONT-FAMILY: times new roman,new york,times,serif"><SPAN
style="FONT-FAMILY: times new roman,new york,times,serif">Gianluca
Arcidiacono</SPAN><BR><BR><BR>
<DIV
style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman,new york,times,serif">-----
Messaggio originale -----<BR>Da: Jayesh Krishna <jayesh@mcs.anl.gov><BR>A:
AGPX <agpxnet@yahoo.it><BR>Cc: mpich-discuss@mcs.anl.gov<BR>Inviato:
Lunedė 12 novembre 2007, 17:29:52<BR>Oggetto: RE: [MPICH] Error handling
issue<BR><BR>
<STYLE type=text/css>DIV {
MARGIN: 0px
}
</STYLE>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=830242716-12112007>Hi,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff size=2><SPAN
class=830242716-12112007> This could probably be an error message given by
the process manager.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=830242716-12112007> How are you aborting the
process?</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=830242716-12112007></SPAN></FONT> </DIV>
<DIV><SPAN class=830242716-12112007><FONT face=Arial color=#0000ff
size=2>Regards,</FONT></SPAN></DIV>
<DIV><SPAN class=830242716-12112007><FONT face=Arial color=#0000ff
size=2>Jayesh</FONT></SPAN></DIV><FONT size=2></FONT><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> owner-mpich-discuss@mcs.anl.gov
[mailto:owner-mpich-discuss@mcs.anl.gov] <B>On Behalf Of
</B>AGPX<BR><B>Sent:</B> Sunday, November 11, 2007 6:37 AM<BR><B>To:</B>
mpich-discuss@mcs.anl.gov<BR><B>Subject:</B> [MPICH] Error handling
issue<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman,new york,times,serif">
<DIV>Hi,<BR><BR>I have write the following code wishing to avoid my main process
to abort on an MPI error:<BR><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Init(&argc,
&argv);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Comm_rank(MPI_COMM_WORLD,
&MPIId);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Comm_size(MPI_COMM_WORLD,
&numprocs);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Comm_set_errhandler(MPI_COMM_WORLD,
<SPAN style="FONT-WEIGHT: bold">MPI_ERRORS_RETURN</SPAN>);</SPAN><BR><BR>but
when I try to terminate a job process on another machine (pcamd3000 is the main
machine, pcamd2600 the other. I use Windows XP Pro on both), then the main
process abort. Here the error message:<BR><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">job
aborted:</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">rank:
node: exit code[: error message]</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">0:
pcamd3000: 1: Fatal error in MPI_Send: Other MPI error, error stack:</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Send(173).............................:
MPI_Send(buf=00B458B0, count=1, MPI_</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">INT,
dest=1, tag=0, comm=0x84000000) failed</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPIDI_CH3I_Progress(148)..................:
handle_sock_op failed</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPIDI_CH3I_Progress_handle_sock_event(497):</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPIDU_Sock_wait(2603).....................:
Il nome di rete specificato non č pių</SPAN><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">
disponibile. (errno 64)</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">1:
pcamd2600: 1: process 1 exited without calling finalize</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">2:
pcamd2600: 1</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><BR>(note
that the message: '<SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">Il
nome di rete specificato non č pių</SPAN><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">
disponibile.' </SPAN>in english is: 'The network name specified is no more
available'.)<BR><BR>What I miss? I have more than one communicator, but I have
used MPI_Comm_set_errhandler as well to set their error handler to
MPI_ERRORS_RETURN. The code is:<BR><BR><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">...</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Group_incl(worldGroup,
nRanks, ranks, &handle.group);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Comm_create(MPI_COMM_WORLD,
handle.group, &handle.comm);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Comm_set_errhandler(handle.comm,
<SPAN style="FONT-WEIGHT: bold">MPI_ERRORS_RETURN</SPAN>);</SPAN><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">...</SPAN><BR><BR>I
have also tried with MPI_Errhandler_set, but this doesn't help:<BR><BR
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif"><SPAN
style="COLOR: rgb(0,0,255); FONT-FAMILY: courier,monaco,monospace,sans-serif">MPI_Errhandler_set(...,
MPI_ERRORS_RETURN);</SPAN><BR><BR>Any suggestion?<BR><BR>Thanks,<BR><BR>-
AGPX<BR><BR><BR></DIV></DIV><BR>
<HR SIZE=1>
<FONT face=Arial size=2>
<HR SIZE=1>
<FONT face=Arial size=2>L'email della prossima generazione? Puoi averla con la
<A
href="http://us.rd.yahoo.com/mail/it/taglines/hotmail/nowyoucan/nextgen/*http://it.docs.yahoo.com/nowyoucan.html"
target=_blank rel=nofollow>nuova Yahoo!
Mail</A></FONT></FONT></DIV><BR></DIV></DIV><BR>
<HR SIZE=1>
<FONT face=Arial size=2>
<HR SIZE=1>
<FONT face=Arial size=2>L'email della prossima generazione? Puoi averla con la
<A
href="http://us.rd.yahoo.com/mail/it/taglines/hotmail/nowyoucan/nextgen/*http://it.docs.yahoo.com/nowyoucan.html">nuova
Yahoo! Mail</A></FONT></FONT></BODY></HTML>