<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><br class=""></div> Trying to solve many "one-dimensional" problems each in parallel on different subset of ranks will be massive pain to do specifically. I recommend just forming a single matrix for all these systems and solving it with KSPSolve and block Jacobi preconditioning or even a parallel direct solver such as with -pc_type lu -pc_factor_mat_solver_type mumps<div class=""><br class=""></div><div class=""> Barry</div><div class=""><br class=""></div><div class="">Yes, this single system, in a certain ordering is block diagonal (each block being tridiagonal) so contains "independent" subsystems; what I suggest above essentially takes advantage of this structure to be reasonably efficient, yet trivial to code.<br class=""><div class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Aug 20, 2021, at 9:14 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">On Fri, Aug 20, 2021 at 7:53 AM Joauma Marichal <<a href="mailto:joauma.marichal@uclouvain.be" class="">joauma.marichal@uclouvain.be</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr" class="">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Dear Sir or Madam,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
I am looking for advice regarding some of PETSc functionnalities. I am currently using PETSc to solve the Navier-Stokes equations on a 3D mesh decomposed over several processors. However, until now, the processors are distributed along the x and z directions
but not along the y one. Indeed, at some point in the algorithm, I must solve a tridiagonal system that depends only on y. Until now, I have therefore performed something like this:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
for(int k = cornp->zs, k<cornp->zs+cornp->zm; ++k){</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
for(int i = cornp->xs, i<cornp->xs+cornp->xm; ++i){</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Create and solve a tridiagonal system for all the y coordinates (which are on the same process)<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
}<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
However, I would like to decompose my mesh in the y direction (as this should improve the code efficiency).</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
I managed to do so by creating a system based on the 3D DM of all my case (so 1 system of size x*y*z). Unfortunately, this does not seem to be very efficient.
<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Do you have some advice on how to cut in the y direction while still being able to solve x*z systems of size y? Should I create 1D DMs?</div></div></blockquote><div class=""><br class=""></div><div class="">1) Are you using a 3D DMDA?</div><div class=""><br class=""></div><div class="">2) Is the coupling much different in the x and z than in the y direction?</div><div class=""><br class=""></div><div class=""> Thanks,</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Thanks a lot for your help. <br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Best regards, <br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
<br class="">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;" class="">
Joauma Marichal<br class="">
</div>
</div>
</blockquote></div><br clear="all" class=""><div class=""><br class=""></div>-- <br class=""><div dir="ltr" class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div><div class=""><br class=""></div><div class=""><a href="http://www.cse.buffalo.edu/~knepley/" target="_blank" class="">https://www.cse.buffalo.edu/~knepley/</a><br class=""></div></div></div></div></div></div></div></div>
</div></blockquote></div><br class=""></div></div></body></html>