<div dir="ltr">I did a PetscBarrier just before calling the vicariate routine and im pretty sure im calling it from every processor, code looks like this:<div><br></div><div><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)">call</span><span style="font-variant-ligatures:no-common-ligatures"> PetscBarrier(PETSC_NULL_OBJECT,ierr)</span></p>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(195,55,32)"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(206,121,36)">print*</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">,</span><span style="font-variant-ligatures:no-common-ligatures">'entering POInit from'</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)">,rank</span></p>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(83,48,225)"><span style="font-variant-ligatures:no-common-ligatures">!call exit()</span></p>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)">call</span><span style="font-variant-ligatures:no-common-ligatures"> </span><span style="font-variant-ligatures:no-common-ligatures;background-color:rgb(230,230,0)">PetscObjsInit</span><span style="font-variant-ligatures:no-common-ligatures">()</span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><font face="arial, helvetica, sans-serif">And output gives:</font></span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><br></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"> entering POInit from 0<br></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures"> entering POInit from 1</span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures"> entering POInit from 2</span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo"><span style="font-variant-ligatures:no-common-ligatures"> entering POInit from 3</span></p></div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures">Still hangs in the same way,</span></div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures">Thanks,</span></div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures">Manuel</span></div><div><span style="font-variant-ligatures:no-common-ligatures"><br></span></div><div><span style="font-variant-ligatures:no-common-ligatures"> </span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 4, 2017 at 2:55 PM, Manuel Valera <span dir="ltr"><<a href="mailto:mvalera@mail.sdsu.edu" target="_blank">mvalera@mail.sdsu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thanks for the answers ! <div><br></div><div>heres the screenshot of what i got from bt in gdb (great hint in how to debug in petsc, didn't know that)</div><div><br></div><div>I don't really know what to look at here, </div><div><br></div><div>Thanks,</div><div><br></div><div>Manuel</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jan 4, 2017 at 2:39 PM, Dave May <span dir="ltr"><<a href="mailto:dave.mayhem23@gmail.com" target="_blank">dave.mayhem23@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Are you certain ALL ranks in PETSC_COMM_WORLD call these function(s). These functions cannot be inside if statements like</div><div>if (rank == 0){</div><div> VecCreateMPI(...)</div><div class="m_2965351358410365906HOEnZb"><div class="m_2965351358410365906h5"><div>}</div><div><br></div><div><br><div class="gmail_quote"><div>On Wed, 4 Jan 2017 at 23:34, Manuel Valera <<a href="mailto:mvalera@mail.sdsu.edu" target="_blank">mvalera@mail.sdsu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="m_2965351358410365906m_396276161700571621gmail_msg">Thanks Dave for the quick answer, appreciate it,<div class="m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621gmail_msg">I just tried that and it didn't make a difference, any other suggestions ?</div><div class="m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621gmail_msg">Thanks,</div><div class="m_2965351358410365906m_396276161700571621gmail_msg">Manuel</div></div><div class="gmail_extra m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621gmail_msg"><div class="gmail_quote m_2965351358410365906m_396276161700571621gmail_msg">On Wed, Jan 4, 2017 at 2:29 PM, Dave May <span class="m_2965351358410365906m_396276161700571621gmail_msg"><<a href="mailto:dave.mayhem23@gmail.com" class="m_2965351358410365906m_396276161700571621gmail_msg" target="_blank">dave.mayhem23@gmail.com</a>></span> wrote:<br class="m_2965351358410365906m_396276161700571621gmail_msg"><blockquote class="gmail_quote m_2965351358410365906m_396276161700571621gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="m_2965351358410365906m_396276161700571621gmail_msg"><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">You need to swap the order of your function calls.</div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Call VecSetSizes() before VecSetType()</div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Thanks,</div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> Dave</div></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380HOEnZb m_2965351358410365906m_396276161700571621gmail_msg"><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380h5 m_2965351358410365906m_396276161700571621gmail_msg"><div class="m_2965351358410365906m_396276161700571621gmail_msg"><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><div class="gmail_quote m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">On Wed, 4 Jan 2017 at 23:21, Manuel Valera <<a href="mailto:mvalera@mail.sdsu.edu" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg" target="_blank">mvalera@mail.sdsu.edu</a>> wrote:<br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><blockquote class="gmail_quote m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Hello all, happy new year,<div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">I'm working on parallelizing my code, it worked and provided some results when i just called more than one processor, but created artifacts because i didn't need one image of the whole program in each processor, conflicting with each other. </div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Since the pressure solver is the main part i need in parallel im chosing mpi to run everything in root processor until its time to solve for pressure, at this point im trying to create a distributed vector using either </div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">call</span><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> VecCreateMPI(PETSC_COMM_WORLD,<wbr>PETSC_DECIDE,nbdp,xp,ierr)</span></p></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">or</span></div><div class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> call</span><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> VecCreate(PETSC_COMM_WORLD,xp,<wbr>ierr); CHKERRQ(ierr)</span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo;color:rgb(83,48,225)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">call</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> VecSetType(xp,VECMPI,ierr) </span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> </span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(52,187,199)" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">call</span><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"> VecSetSizes(xp,PETSC_DECIDE,nb<wbr>dp,ierr); CHKERRQ(ierr)</span><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></span></p><p style="margin:0px;font-size:11px;line-height:normal;font-family:menlo" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></span></p><p style="margin:0px;line-height:normal" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><font face="arial, helvetica, sans-serif" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">In both cases program hangs at this point, something it never happened on the naive way i described before. I've made sure the global size, nbdp, is the same in every processor. What can be wrong?</font></span></p><p style="margin:0px;line-height:normal" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><font face="arial, helvetica, sans-serif" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></font></span></p><p style="margin:0px;line-height:normal" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><font face="arial, helvetica, sans-serif" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Thanks for your kind help,</font></span></p><p style="margin:0px;line-height:normal" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><font face="arial, helvetica, sans-serif" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></font></span></p><p style="margin:0px;line-height:normal" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><span style="font-variant-ligatures:no-common-ligatures" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><font face="arial, helvetica, sans-serif" class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg">Manuel.</font></span></p></span></div></div><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"><br class="m_2965351358410365906m_396276161700571621m_3699639947378775380m_2673180215534198650gmail_msg m_2965351358410365906m_396276161700571621gmail_msg"></blockquote></div></div></div><br><br></div></div></blockquote></div><br class="m_2965351358410365906m_396276161700571621gmail_msg"></div><br><br></blockquote></div></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>