<div>Hello</div><div><br></div>Pavan and Nicolas, thank you very much, I see the point in making even the smallest error fatal, i just have a little trouble understanding the error landscape by an large.<div><br></div><div>
I mean, i doubt that mpi error handler will handle errors that are not coming from mpi context. </div><div><br></div><div>For example, i painstakingly debugged all my mpi calls, so lets asume its all quite on the MPI front, but i cant wouch for a certain library that does minor work that has nothing to do with message passing.</div>
<div><br></div><div>I dont want my node calling all rank abort if it segfaults, i mean, if the cause of the error would have nothing to do with mpi, how can i safeguard myself from such troubles?<br><div><br></div><div>Eugene</div>
<div><br><br><div class="gmail_quote">2011/1/29 Pavan Balaji <span dir="ltr"><<a href="mailto:balaji@mcs.anl.gov">balaji@mcs.anl.gov</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
There are also some notes in the README in 1.3.2rc1 describing how to use errors returned by MPI functions, what you can expect, and what you can't.<br>
<br>
-- Pavan<div><div></div><div class="h5"><br>
<br>
On 01/29/2011 12:06 PM, Nicolas Rosner wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Let's suppose you're running a simple MPI program, one communicator,<br>
ten ranks or so. Now imagine rank 7 hits an off-by-one bug, trespasses<br>
the end of some array and segfaults.<br>
<br>
If, by default, your whole program dies immediately, then what? You<br>
look at the logs, think, insert a few printfs, then track the the<br>
off-by-one in a couple of minutes.<br>
<br>
If instead the rest just moves on with a dead rank 7, you end up with<br>
a half-dead system that will eventually collapse anyway, misleading<br>
symptoms and a tenfold increase in solution time. Worse, it might even<br>
not collapse, hiding a bug that will be much harder to track down and<br>
fix in the future when you don't even remember writing that code.<br>
<br>
MPICH2 allows you to implement a certain level of runtime fault<br>
tolerance; I hear future versions will allow a lot more. But<br>
remember: there is no free lunch -- if you want to write a robust<br>
system, you'll need to write error handlers that actually handle<br>
errors robustly.<br>
<br>
Until you do so, keeping all local fatal errors globally fatal is<br>
wise. My .02, at least.<br>
<br>
(Try looking up MPI_ERRORS_ARE_FATAL.)<br>
<br>
Regards,<br>
Nicolás<br>
<br>
<br>
<br>
On Sat, Jan 29, 2011 at 5:54 AM, Eugene N<<a href="mailto:neverov.biks.07.1@gmail.com" target="_blank">neverov.biks.07.1@gmail.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi<br>
is it true that even if my most humble mpich2 client node will abort, all my<br>
claster will go down? How can i cure it?<br>
Thanks,<br>
Eugene<br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
<br>
<br>
</blockquote>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</blockquote>
<br></div></div><font color="#888888">
-- <br>
Pavan Balaji<br>
<a href="http://www.mcs.anl.gov/~balaji" target="_blank">http://www.mcs.anl.gov/~balaji</a></font><div><div></div><div class="h5"><br>
_______________________________________________<br>
mpich-discuss mailing list<br>
<a href="mailto:mpich-discuss@mcs.anl.gov" target="_blank">mpich-discuss@mcs.anl.gov</a><br>
<a href="https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/mpich-discuss</a><br>
</div></div></blockquote></div><br></div></div>