<span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">I am using MPI_Comm_spawn to dynamically run workers. However, when the workers exit they get hung up on MPI_Finalize. Here is a short program which shows the issue...</span><br>
<font class="Apple-style-span" face="arial, sans-serif"><br></font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">It responds to several commands...</span><br><font class="Apple-style-span" face="arial, sans-serif"><br>
</font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">Do</span><br><font class="Apple-style-span" face="arial, sans-serif"><br></font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">start</span><br>
<span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">stop</span><br><font class="Apple-style-span" face="arial, sans-serif"><br></font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">and then check how many processes are running - it should be 1, not 2.</span><br>
<font class="Apple-style-span" face="arial, sans-serif"><br></font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">I am using MPICH2 1.4.1-p1.</span><br><font class="Apple-style-span" face="arial, sans-serif"><br>
</font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">Thanks,</span><br><font class="Apple-style-span" face="arial, sans-serif"><br></font><span class="Apple-style-span" style="font-size: 13px; background-color: rgb(255, 255, 255); ">Jon</span><br>
<font class="Apple-style-span" face="'courier new', monospace"><br></font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">#include <sys/types.h></span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">#include <unistd.h></span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">#include <iostream></span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">#include "mpi.h"</span><br><font class="Apple-style-span" face="'courier new', monospace"><br>
</font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">using namespace std;</span><br><font class="Apple-style-span" face="'courier new', monospace"><br>
<br></font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">main(int argc, char **argv)</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">{</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Init(&argc, &argv);</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Comm parent;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Comm_get_parent(&parent);</span><br><font class="Apple-style-span" face="'courier new', monospace"><br>
</font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> // Master</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (parent == MPI_COMM_NULL) {</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << getpid() << endl;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Comm intercom = MPI_COMM_NULL;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> while (1) {</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << "Enter: ";</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> string s;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cin >> s;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (s == "start") {</span><br><span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">if (intercom != MPI_COMM_NULL) {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << "already started" << endl;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> continue;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">}</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">MPI_Comm_spawn(argv[0], MPI_ARGV_NULL, 1, MPI_INFO_NULL, 0, MPI_COMM_SELF, &intercom, MPI_ERRCODES_IGNORE);</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">continue;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (s == "stop") {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">if (intercom == MPI_COMM_NULL) {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << "worker not running" << endl;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> continue;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">}</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">MPI_Send(const_cast<char*>(s.</span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">c_str()), s.size(), MPI_CHAR, 0, 0, intercom);</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">intercom = MPI_COMM_NULL;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">//</span><span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">MPI_Finalize(); // This will allow the workers to die, but then I can not restart them.</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">continue;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (s == "exit") {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">if (intercom != MPI_COMM_NULL) {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << "need to stop before exit" << endl;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> continue;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">}</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">break;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (intercom == MPI_COMM_NULL) {</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">cout << "need to start" << endl;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">continue;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Send(const_cast<char*>(s.</span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">c_str()), s.size(), MPI_CHAR, 0, 0, intercom);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> char buf[1000];</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Status status;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Recv(buf, 1000, MPI_CHAR, MPI_ANY_SOURCE, MPI_ANY_TAG, intercom, &status);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> int count;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Get_count(&status, MPI_CHAR, &count);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> buf[count] = 0;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> string t = buf;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> cout << "worker returned " << t << endl;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br>
<font class="Apple-style-span" face="'courier new', monospace"><br></font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> // Worker</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (parent != MPI_COMM_NULL) {</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> while (1) {</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> char buf[1000];</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Status status;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Recv(buf, 1000, MPI_CHAR, MPI_ANY_SOURCE, MPI_ANY_TAG, parent, &status);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> int count;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Get_count(&status, MPI_CHAR, &count);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> buf[count] = 0;</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> string s = buf;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> if (s == "stop") {</span><br><span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">cout << "worker stopping" << endl;</span><br>
<span style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); white-space: pre-wrap; "> </span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">break;</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Send(const_cast<char*>(s.</span><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); ">c_str()), s.size(), MPI_CHAR, 0, 0, parent);</span><br>
<span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> }</span><br>
<font class="Apple-style-span" face="'courier new', monospace"><br></font><span class="Apple-style-span" style="font-family: 'courier new', monospace; font-size: 13px; background-color: rgb(255, 255, 255); "> MPI_Finalize();</span><br>
<div><div><div style="font-family: arial, sans-serif; "><font face="'courier new', monospace">}</font></div></div></div><div><div><div><div><font face="'courier new', monospace"><br></font></div></div></div>
</div>